You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2016/07/16 03:17:13 UTC

[JENKINS] Lucene-Solr-NightlyTests-6.x - Build # 121 - Still Failing

Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-6.x/121/

4 tests failed.
FAILED:  org.apache.solr.cloud.LeaderFailoverAfterPartitionTest.test

Error Message:
Expected 2 of 3 replicas to be active but only found 1; [core_node2:{"core":"c8n_1x3_lf_shard1_replica1","base_url":"http://127.0.0.1:55779","node_name":"127.0.0.1:55779_","state":"active","leader":"true"}]; clusterState: DocCollection(c8n_1x3_lf//collections/c8n_1x3_lf/state.json/20)={   "replicationFactor":"3",   "shards":{"shard1":{       "range":"80000000-7fffffff",       "state":"active",       "replicas":{         "core_node1":{           "state":"down",           "base_url":"http://127.0.0.1:54909",           "core":"c8n_1x3_lf_shard1_replica3",           "node_name":"127.0.0.1:54909_"},         "core_node2":{           "core":"c8n_1x3_lf_shard1_replica1",           "base_url":"http://127.0.0.1:55779",           "node_name":"127.0.0.1:55779_",           "state":"active",           "leader":"true"},         "core_node3":{           "core":"c8n_1x3_lf_shard1_replica2",           "base_url":"http://127.0.0.1:38009",           "node_name":"127.0.0.1:38009_",           "state":"down"}}}},   "router":{"name":"compositeId"},   "maxShardsPerNode":"1",   "autoAddReplicas":"false"}

Stack Trace:
java.lang.AssertionError: Expected 2 of 3 replicas to be active but only found 1; [core_node2:{"core":"c8n_1x3_lf_shard1_replica1","base_url":"http://127.0.0.1:55779","node_name":"127.0.0.1:55779_","state":"active","leader":"true"}]; clusterState: DocCollection(c8n_1x3_lf//collections/c8n_1x3_lf/state.json/20)={
  "replicationFactor":"3",
  "shards":{"shard1":{
      "range":"80000000-7fffffff",
      "state":"active",
      "replicas":{
        "core_node1":{
          "state":"down",
          "base_url":"http://127.0.0.1:54909",
          "core":"c8n_1x3_lf_shard1_replica3",
          "node_name":"127.0.0.1:54909_"},
        "core_node2":{
          "core":"c8n_1x3_lf_shard1_replica1",
          "base_url":"http://127.0.0.1:55779",
          "node_name":"127.0.0.1:55779_",
          "state":"active",
          "leader":"true"},
        "core_node3":{
          "core":"c8n_1x3_lf_shard1_replica2",
          "base_url":"http://127.0.0.1:38009",
          "node_name":"127.0.0.1:38009_",
          "state":"down"}}}},
  "router":{"name":"compositeId"},
  "maxShardsPerNode":"1",
  "autoAddReplicas":"false"}
	at __randomizedtesting.SeedInfo.seed([53C8CD85C44C8CD3:DB9CF25F6AB0E12B]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.apache.solr.cloud.LeaderFailoverAfterPartitionTest.testRf3WithLeaderFailover(LeaderFailoverAfterPartitionTest.java:168)
	at org.apache.solr.cloud.LeaderFailoverAfterPartitionTest.test(LeaderFailoverAfterPartitionTest.java:55)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.handler.TestReqParamsAPI.test

Error Message:
Could not get expected value  'CY val' for path 'params/c' full output: {   "responseHeader":{     "status":0,     "QTime":0},   "params":{     "a":"A val",     "b":"B val",     "wt":"json",     "useParams":""},   "context":{     "webapp":"",     "path":"/dump1",     "httpMethod":"GET"}},  from server:  https://127.0.0.1:42946/collection1

Stack Trace:
java.lang.AssertionError: Could not get expected value  'CY val' for path 'params/c' full output: {
  "responseHeader":{
    "status":0,
    "QTime":0},
  "params":{
    "a":"A val",
    "b":"B val",
    "wt":"json",
    "useParams":""},
  "context":{
    "webapp":"",
    "path":"/dump1",
    "httpMethod":"GET"}},  from server:  https://127.0.0.1:42946/collection1
	at __randomizedtesting.SeedInfo.seed([53C8CD85C44C8CD3:DB9CF25F6AB0E12B]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.apache.solr.core.TestSolrConfigHandler.testForResponseElement(TestSolrConfigHandler.java:481)
	at org.apache.solr.handler.TestReqParamsAPI.testReqParams(TestReqParamsAPI.java:171)
	at org.apache.solr.handler.TestReqParamsAPI.test(TestReqParamsAPI.java:61)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.test

Error Message:
Timeout occured while waiting response from server at: https://127.0.0.1:44842

Stack Trace:
org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: https://127.0.0.1:44842
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:601)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:259)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:248)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.makeRequest(CollectionsAPIDistributedZkTest.java:399)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testErrorHandling(CollectionsAPIDistributedZkTest.java:515)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test(CollectionsAPIDistributedZkTest.java:179)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.net.SocketTimeoutException: Read timed out
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
	at java.net.SocketInputStream.read(SocketInputStream.java:170)
	at java.net.SocketInputStream.read(SocketInputStream.java:141)
	at sun.security.ssl.InputRecord.readFully(InputRecord.java:465)
	at sun.security.ssl.InputRecord.read(InputRecord.java:503)
	at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:973)
	at sun.security.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:930)
	at sun.security.ssl.AppInputStream.read(AppInputStream.java:105)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:160)
	at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:84)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:273)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
	at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
	at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)
	at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)
	at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197)
	at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)
	at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)
	at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685)
	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487)
	at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:495)
	... 47 more


FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=30486, name=collection2, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=30486, name=collection2, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
	at __randomizedtesting.SeedInfo.seed([53C8CD85C44C8CD3:DB9CF25F6AB0E12B]:0)
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:36282: collection already exists: awholynewstresscollection_collection2_1
	at __randomizedtesting.SeedInfo.seed([53C8CD85C44C8CD3]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:590)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:259)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:248)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:403)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:356)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1270)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1040)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:976)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1599)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1620)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:987)




Build Log:
[...truncated 11212 lines...]
   [junit4] Suite: org.apache.solr.cloud.LeaderFailoverAfterPartitionTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_53C8CD85C44C8CD3-001/init-core-data-001
   [junit4]   2> 254836 INFO  (SUITE-LeaderFailoverAfterPartitionTest-seed#[53C8CD85C44C8CD3]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=https://issues.apache.org/jira/browse/SOLR-5776)
   [junit4]   2> 254836 INFO  (SUITE-LeaderFailoverAfterPartitionTest-seed#[53C8CD85C44C8CD3]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   2> 254841 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 254842 INFO  (Thread-787) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 254842 INFO  (Thread-787) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 254942 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [    ] o.a.s.c.ZkTestServer start zk server on port:51810
   [junit4]   2> 254942 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 254942 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 254947 INFO  (zkCallback-168-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@735b24ba name:ZooKeeperConnection Watcher:127.0.0.1:51810 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 254947 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 254947 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 254948 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 254951 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 254951 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 254957 INFO  (zkCallback-169-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@35dd0133 name:ZooKeeperConnection Watcher:127.0.0.1:51810/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 254957 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 254958 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 254958 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 254959 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 254961 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 254962 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 254963 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 254963 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 254965 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 254965 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 254967 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 254967 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 254968 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 254968 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 254969 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 254970 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 254971 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 254971 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 254972 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 254972 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 254973 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 254973 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 254975 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 254975 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 254976 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 254976 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 254977 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 254977 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 255077 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_53C8CD85C44C8CD3-001/control-001/cores/collection1
   [junit4]   2> 255080 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [    ] o.e.j.s.Server jetty-9.3.8.v20160314
   [junit4]   2> 255091 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@9ace822{/,null,AVAILABLE}
   [junit4]   2> 255094 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [    ] o.e.j.s.ServerConnector Started ServerConnector@18819dfe{HTTP/1.1,[http/1.1]}{127.0.0.1:34115}
   [junit4]   2> 255094 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [    ] o.e.j.s.Server Started @259593ms
   [junit4]   2> 255094 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_53C8CD85C44C8CD3-001/tempDir-001/control/data, hostContext=/, hostPort=38009, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_53C8CD85C44C8CD3-001/control-001/cores}
   [junit4]   2> 255094 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@73d16e93
   [junit4]   2> 255094 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_53C8CD85C44C8CD3-001/control-001'
   [junit4]   2> 255094 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 255094 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 255095 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 255097 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 255099 INFO  (zkCallback-170-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1ed314a6 name:ZooKeeperConnection Watcher:127.0.0.1:51810/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 255099 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 255099 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 255102 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 255102 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_53C8CD85C44C8CD3-001/control-001/solr.xml
   [junit4]   2> 255107 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_53C8CD85C44C8CD3-001/control-001/cores
   [junit4]   2> 255107 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [    ] o.a.s.c.CoreContainer New CoreContainer 137092164
   [junit4]   2> 255107 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_53C8CD85C44C8CD3-001/control-001]
   [junit4]   2> 255107 WARN  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_53C8CD85C44C8CD3-001/control-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_53C8CD85C44C8CD3-001/control-001/lib
   [junit4]   2> 255108 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,connectionsEvictorSleepDelay : 5000,maxConnectionIdleTime : 40000,
   [junit4]   2> 255113 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 255114 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 255114 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 255114 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:51810/solr
   [junit4]   2> 255114 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 255114 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 255115 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 255119 INFO  (zkCallback-173-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@498cb511 name:ZooKeeperConnection Watcher:127.0.0.1:51810 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 255119 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 255119 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 255120 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [n:127.0.0.1:38009_    ] o.a.s.c.ZkController Added new OnReconnect listener org.apache.solr.cloud.ZkController$$Lambda$18/1651980046@6df0566
   [junit4]   2> 255125 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [n:127.0.0.1:38009_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 255127 INFO  (zkCallback-174-thread-1-processing-n:127.0.0.1:38009_) [n:127.0.0.1:38009_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@634f652d name:ZooKeeperConnection Watcher:127.0.0.1:51810/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 255127 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [n:127.0.0.1:38009_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 255128 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [n:127.0.0.1:38009_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 255129 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [n:127.0.0.1:38009_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 255131 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [n:127.0.0.1:38009_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 255133 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [n:127.0.0.1:38009_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 255134 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [n:127.0.0.1:38009_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 255136 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [n:127.0.0.1:38009_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 255137 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [n:127.0.0.1:38009_    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 255138 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [n:127.0.0.1:38009_    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 255139 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [n:127.0.0.1:38009_    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 255139 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [n:127.0.0.1:38009_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 255140 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [n:127.0.0.1:38009_    ] o.a.s.c.c.ZkStateReader Loaded empty cluster properties
   [junit4]   2> 255140 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [n:127.0.0.1:38009_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (0)
   [junit4]   2> 255143 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [n:127.0.0.1:38009_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 255143 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [n:127.0.0.1:38009_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 255144 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [n:127.0.0.1:38009_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 255145 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [n:127.0.0.1:38009_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/96248157586980868-127.0.0.1:38009_-n_0000000000
   [junit4]   2> 255145 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [n:127.0.0.1:38009_    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:38009_
   [junit4]   2> 255145 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [n:127.0.0.1:38009_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 255146 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [n:127.0.0.1:38009_    ] o.a.s.c.Overseer Overseer (id=96248157586980868-127.0.0.1:38009_-n_0000000000) starting
   [junit4]   2> 255148 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [n:127.0.0.1:38009_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 255152 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [n:127.0.0.1:38009_    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 255154 INFO  (OverseerStateUpdate-96248157586980868-127.0.0.1:38009_-n_0000000000) [n:127.0.0.1:38009_    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 255157 INFO  (OverseerCollectionConfigSetProcessor-96248157586980868-127.0.0.1:38009_-n_0000000000) [n:127.0.0.1:38009_    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 255159 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [n:127.0.0.1:38009_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:38009_
   [junit4]   2> 255159 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [n:127.0.0.1:38009_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:38009_
   [junit4]   2> 255159 INFO  (OverseerStateUpdate-96248157586980868-127.0.0.1:38009_-n_0000000000) [n:127.0.0.1:38009_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (0)
   [junit4]   2> 255161 INFO  (zkCallback-174-thread-1-processing-n:127.0.0.1:38009_) [n:127.0.0.1:38009_    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [0])
   [junit4]   2> 255161 INFO  (zkCallback-174-thread-1-processing-n:127.0.0.1:38009_) [n:127.0.0.1:38009_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 255174 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [n:127.0.0.1:38009_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 255174 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [n:127.0.0.1:38009_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 255175 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [n:127.0.0.1:38009_    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_53C8CD85C44C8CD3-001/control-001/cores
   [junit4]   2> 255176 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [n:127.0.0.1:38009_    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, loadOnStartup=true, configSetProperties=configsetprops.json, collection=control_collection, coreNodeName=, dataDir=data/, shard=}
   [junit4]   2> 255176 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [n:127.0.0.1:38009_    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_53C8CD85C44C8CD3-001/control-001/cores/collection1
   [junit4]   2> 255176 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [n:127.0.0.1:38009_    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 255178 INFO  (coreLoadExecutor-465-thread-1-processing-n:127.0.0.1:38009_) [n:127.0.0.1:38009_ c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 255178 INFO  (coreLoadExecutor-465-thread-1-processing-n:127.0.0.1:38009_) [n:127.0.0.1:38009_ c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 255179 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [n:127.0.0.1:38009_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2
   [junit4]   2> 255179 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [n:127.0.0.1:38009_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 255179 INFO  (coreLoadExecutor-465-thread-1-processing-n:127.0.0.1:38009_) [n:127.0.0.1:38009_ c:control_collection   x:collection1] o.a.s.c.ZkController Collection control_collection not visible yet, but flagging it so a watch is registered when it becomes visible
   [junit4]   2> 255179 INFO  (OverseerStateUpdate-96248157586980868-127.0.0.1:38009_-n_0000000000) [n:127.0.0.1:38009_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:38009",
   [junit4]   2>   "node_name":"127.0.0.1:38009_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 255180 INFO  (OverseerStateUpdate-96248157586980868-127.0.0.1:38009_-n_0000000000) [n:127.0.0.1:38009_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:38009",
   [junit4]   2>   "node_name":"127.0.0.1:38009_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 255180 INFO  (OverseerStateUpdate-96248157586980868-127.0.0.1:38009_-n_0000000000) [n:127.0.0.1:38009_    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 255181 INFO  (OverseerStateUpdate-96248157586980868-127.0.0.1:38009_-n_0000000000) [n:127.0.0.1:38009_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 255187 INFO  (coreLoadExecutor-465-thread-1-processing-n:127.0.0.1:38009_) [n:127.0.0.1:38009_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Deleting data for [control_collection]
   [junit4]   2> 255187 INFO  (zkCallback-174-thread-1-processing-n:127.0.0.1:38009_) [n:127.0.0.1:38009_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 255189 INFO  (coreLoadExecutor-465-thread-1-processing-n:127.0.0.1:38009_) [n:127.0.0.1:38009_ c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 255189 INFO  (coreLoadExecutor-465-thread-1-processing-n:127.0.0.1:38009_) [n:127.0.0.1:38009_ c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 255189 INFO  (coreLoadExecutor-465-thread-1-processing-n:127.0.0.1:38009_) [n:127.0.0.1:38009_ c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 255189 INFO  (coreLoadExecutor-465-thread-1-processing-n:127.0.0.1:38009_) [n:127.0.0.1:38009_ c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 255189 INFO  (coreLoadExecutor-465-thread-1-processing-n:127.0.0.1:38009_) [n:127.0.0.1:38009_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/control_collection]
   [junit4]   2> 255190 INFO  (coreLoadExecutor-465-thread-1-processing-n:127.0.0.1:38009_) [n:127.0.0.1:38009_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/control_collection] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 255190 INFO  (coreLoadExecutor-465-thread-1-processing-n:127.0.0.1:38009_) [n:127.0.0.1:38009_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_53C8CD85C44C8CD3-001/control-001/cores/collection1'
   [junit4]   2> 255190 INFO  (coreLoadExecutor-465-thread-1-processing-n:127.0.0.1:38009_) [n:127.0.0.1:38009_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 255190 INFO  (coreLoadExecutor-465-thread-1-processing-n:127.0.0.1:38009_) [n:127.0.0.1:38009_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 255191 INFO  (coreLoadExecutor-465-thread-1-processing-n:127.0.0.1:38009_) [n:127.0.0.1:38009_ c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 255200 INFO  (coreLoadExecutor-465-thread-1-processing-n:127.0.0.1:38009_) [n:127.0.0.1:38009_ c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 255204 WARN  (coreLoadExecutor-465-thread-1-processing-n:127.0.0.1:38009_) [n:127.0.0.1:38009_ c:control_collection   x:collection1] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
   [junit4]   2> 255204 INFO  (coreLoadExecutor-465-thread-1-processing-n:127.0.0.1:38009_) [n:127.0.0.1:38009_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.2.0
   [junit4]   2> 255213 INFO  (coreLoadExecutor-465-thread-1-processing-n:127.0.0.1:38009_) [n:127.0.0.1:38009_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 255219 INFO  (coreLoadExecutor-465-thread-1-processing-n:127.0.0.1:38009_) [n:127.0.0.1:38009_ c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 255304 INFO  (coreLoadExecutor-465-thread-1-processing-n:127.0.0.1:38009_) [n:127.0.0.1:38009_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 255310 INFO  (coreLoadExecutor-465-thread-1-processing-n:127.0.0.1:38009_) [n:127.0.0.1:38009_ c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 255311 INFO  (coreLoadExecutor-465-thread-1-processing-n:127.0.0.1:38009_) [n:127.0.0.1:38009_ c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 255327 INFO  (coreLoadExecutor-465-thread-1-processing-n:127.0.0.1:38009_) [n:127.0.0.1:38009_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 255329 INFO  (coreLoadExecutor-465-thread-1-processing-n:127.0.0.1:38009_) [n:127.0.0.1:38009_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 255331 INFO  (coreLoadExecutor-465-thread-1-processing-n:127.0.0.1:38009_) [n:127.0.0.1:38009_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 255331 INFO  (coreLoadExecutor-465-thread-1-processing-n:127.0.0.1:38009_) [n:127.0.0.1:38009_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 255333 INFO  (coreLoadExecutor-465-thread-1-processing-n:127.0.0.1:38009_) [n:127.0.0.1:38009_ c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2
   [junit4]   2> 255333 INFO  (coreLoadExecutor-465-thread-1-processing-n:127.0.0.1:38009_) [n:127.0.0.1:38009_ c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 255333 INFO  (coreLoadExecutor-465-thread-1-processing-n:127.0.0.1:38009_) [n:127.0.0.1:38009_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 255333 INFO  (coreLoadExecutor-465-thread-1-processing-n:127.0.0.1:38009_) [n:127.0.0.1:38009_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_53C8CD85C44C8CD3-001/control-001/cores/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_53C8CD85C44C8CD3-001/control-001/cores/collection1/data/]
   [junit4]   2> 255333 INFO  (coreLoadExecutor-465-thread-1-processing-n:127.0.0.1:38009_) [n:127.0.0.1:38009_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@514d7b2d
   [junit4]   2> 255334 INFO  (coreLoadExecutor-465-thread-1-processing-n:127.0.0.1:38009_) [n:127.0.0.1:38009_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_53C8CD85C44C8CD3-001/control-001/cores/collection1/data
   [junit4]   2> 255334 INFO  (coreLoadExecutor-465-thread-1-processing-n:127.0.0.1:38009_) [n:127.0.0.1:38009_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_53C8CD85C44C8CD3-001/control-001/cores/collection1/data/index/
   [junit4]   2> 255334 WARN  (coreLoadExecutor-465-thread-1-processing-n:127.0.0.1:38009_) [n:127.0.0.1:38009_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_53C8CD85C44C8CD3-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 255334 INFO  (coreLoadExecutor-465-thread-1-processing-n:127.0.0.1:38009_) [n:127.0.0.1:38009_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_53C8CD85C44C8CD3-001/control-001/cores/collection1/data/index
   [junit4]   2> 255335 INFO  (coreLoadExecutor-465-thread-1-processing-n:127.0.0.1:38009_) [n:127.0.0.1:38009_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=33, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=0.8671875, noCFSRatio=0.8698527042071156]
   [junit4]   2> 255335 INFO  (coreLoadExecutor-465-thread-1-processing-n:127.0.0.1:38009_) [n:127.0.0.1:38009_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@660b12d1 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@3c3b610d),segFN=segments_1,generation=1}
   [junit4]   2> 255335 INFO  (coreLoadExecutor-465-thread-1-processing-n:127.0.0.1:38009_) [n:127.0.0.1:38009_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 255344 INFO  (coreLoadExecutor-465-thread-1-processing-n:127.0.0.1:38009_) [n:127.0.0.1:38009_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 255344 INFO  (coreLoadExecutor-465-thread-1-processing-n:127.0.0.1:38009_) [n:127.0.0.1:38009_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 255344 INFO  (coreLoadExecutor-465-thread-1-processing-n:127.0.0.1:38009_) [n:127.0.0.1:38009_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 255344 INFO  (coreLoadExecutor-465-thread-1-processing-n:127.0.0.1:38009_) [n:127.0.0.1:38009_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 255345 INFO  (coreLoadExecutor-465-thread-1-processing-n:127.0.0.1:38009_) [n:127.0.0.1:38009_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 255345 INFO  (coreLoadExecutor-465-thread-1-processing-n:127.0.0.1:38009_) [n:127.0.0.1:38009_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 255345 INFO  (coreLoadExecutor-465-thread-1-processing-n:127.0.0.1:38009_) [n:127.0.0.1:38009_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 255345 INFO  (coreLoadExecutor-465-thread-1-processing-n:127.0.0.1:38009_) [n:127.0.0.1:38009_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 255345 INFO  (coreLoadExecutor-465-thread-1-processing-n:127.0.0.1:38009_) [n:127.0.0.1:38009_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 255345 WARN  (coreLoadExecutor-465-thread-1-processing-n:127.0.0.1:38009_) [n:127.0.0.1:38009_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
   [junit4]   2> 255347 INFO  (coreLoadExecutor-465-thread-1-processing-n:127.0.0.1:38009_) [n:127.0.0.1:38009_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 255348 INFO  (coreLoadExecutor-465-thread-1-processing-n:127.0.0.1:38009_) [n:127.0.0.1:38009_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 255349 INFO  (coreLoadExecutor-465-thread-1-processing-n:127.0.0.1:38009_) [n:127.0.0.1:38009_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 255349 INFO  (coreLoadExecutor-465-thread-1-processing-n:127.0.0.1:38009_) [n:127.0.0.1:38009_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 255351 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:38009_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_53C8CD85C44C8CD3-001/control-001/cores/collection1/data/
   [junit4]   2> 255353 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:38009_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_53C8CD85C44C8CD3-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 255354 INFO  (coreLoadExecutor-465-thread-1-processing-n:127.0.0.1:38009_) [n:127.0.0.1:38009_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/mbeans,standard,/update/json/docs,/admin/luke,/export,/get,/admin/properties,/update/json,/admin/threads,/dump,/update/csv,/sql,/graph,/admin/segments,/admin/system,/replication,/config,/stream,/schema,/admin/plugins,/admin/logging,/admin/ping,/update,/admin/file,/terms
   [junit4]   2> 255355 INFO  (coreLoadExecutor-465-thread-1-processing-n:127.0.0.1:38009_) [n:127.0.0.1:38009_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 255357 INFO  (coreLoadExecutor-465-thread-1-processing-n:127.0.0.1:38009_) [n:127.0.0.1:38009_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 255357 INFO  (coreLoadExecutor-465-thread-1-processing-n:127.0.0.1:38009_) [n:127.0.0.1:38009_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 255358 INFO  (coreLoadExecutor-465-thread-1-processing-n:127.0.0.1:38009_) [n:127.0.0.1:38009_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 255358 INFO  (coreLoadExecutor-465-thread-1-processing-n:127.0.0.1:38009_) [n:127.0.0.1:38009_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 255358 INFO  (coreLoadExecutor-465-thread-1-processing-n:127.0.0.1:38009_) [n:127.0.0.1:38009_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=28, maxMergeAtOnceExplicit=38, maxMergedSegmentMB=19.552734375, floorSegmentMB=1.501953125, forceMergeDeletesPctAllowed=17.826478631557844, segmentsPerTier=50.0, maxCFSSegmentSizeMB=1.859375, noCFSRatio=0.8859658523399524
   [junit4]   2> 255359 INFO  (coreLoadExecutor-465-thread-1-processing-n:127.0.0.1:38009_) [n:127.0.0.1:38009_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@660b12d1 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@3c3b610d),segFN=segments_1,generation=1}
   [junit4]   2> 255359 INFO  (coreLoadExecutor-465-thread-1-processing-n:127.0.0.1:38009_) [n:127.0.0.1:38009_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 255360 INFO  (coreLoadExecutor-465-thread-1-processing-n:127.0.0.1:38009_) [n:127.0.0.1:38009_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@4c24c776[collection1] main]
   [junit4]   2> 255360 INFO  (coreLoadExecutor-465-thread-1-processing-n:127.0.0.1:38009_) [n:127.0.0.1:38009_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/control_collection]
   [junit4]   2> 255361 INFO  (coreLoadExecutor-465-thread-1-processing-n:127.0.0.1:38009_) [n:127.0.0.1:38009_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/control_collection] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 255361 INFO  (coreLoadExecutor-465-thread-1-processing-n:127.0.0.1:38009_) [n:127.0.0.1:38009_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 255362 INFO  (coreLoadExecutor-465-thread-1-processing-n:127.0.0.1:38009_) [n:127.0.0.1:38009_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 255362 INFO  (coreLoadExecutor-465-thread-1-processing-n:127.0.0.1:38009_) [n:127.0.0.1:38009_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 255362 INFO  (coreLoadExecutor-465-thread-1-processing-n:127.0.0.1:38009_) [n:127.0.0.1:38009_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 255362 INFO  (coreLoadExecutor-465-thread-1-processing-n:127.0.0.1:38009_) [n:127.0.0.1:38009_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 255362 INFO  (coreLoadExecutor-465-thread-1-processing-n:127.0.0.1:38009_) [n:127.0.0.1:38009_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 255362 INFO  (coreLoadExecutor-465-thread-1-processing-n:127.0.0.1:38009_) [n:127.0.0.1:38009_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 255362 INFO  (coreLoadExecutor-465-thread-1-processing-n:127.0.0.1:38009_) [n:127.0.0.1:38009_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 255363 INFO  (coreLoadExecutor-465-thread-1-processing-n:127.0.0.1:38009_) [n:127.0.0.1:38009_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 255363 INFO  (coreLoadExecutor-465-thread-1-processing-n:127.0.0.1:38009_) [n:127.0.0.1:38009_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 255363 INFO  (coreLoadExecutor-465-thread-1-processing-n:127.0.0.1:38009_) [n:127.0.0.1:38009_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 255363 INFO  (coreLoadExecutor-465-thread-1-processing-n:127.0.0.1:38009_) [n:127.0.0.1:38009_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1539970521933807616
   [junit4]   2> 255365 INFO  (coreLoadExecutor-465-thread-1-processing-n:127.0.0.1:38009_) [n:127.0.0.1:38009_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 1.0ms to seed version buckets with highest version 1539970521933807616
   [junit4]   2> 255365 INFO  (coreLoadExecutor-465-thread-1-processing-n:127.0.0.1:38009_) [n:127.0.0.1:38009_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 255366 INFO  (searcherExecutor-466-thread-1-processing-n:127.0.0.1:38009_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:38009_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@4c24c776[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 255369 INFO  (coreLoadExecutor-465-thread-1-processing-n:127.0.0.1:38009_) [n:127.0.0.1:38009_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 255373 INFO  (coreZkRegister-459-thread-1-processing-n:127.0.0.1:38009_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:38009_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:38009 collection:control_collection shard:shard1
   [junit4]   2> 255378 INFO  (coreZkRegister-459-thread-1-processing-n:127.0.0.1:38009_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:38009_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 255438 INFO  (coreZkRegister-459-thread-1-processing-n:127.0.0.1:38009_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:38009_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/96248157586980868-core_node1-n_0000000000
   [junit4]   2> 255445 INFO  (coreZkRegister-459-thread-1-processing-n:127.0.0.1:38009_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:38009_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard=shard1 and weAreReplacement=false and leaderVoteWait=10000
   [junit4]   2> 255473 INFO  (coreZkRegister-459-thread-1-processing-n:127.0.0.1:38009_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:38009_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 255473 INFO  (coreZkRegister-459-thread-1-processing-n:127.0.0.1:38009_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:38009_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 255473 INFO  (coreZkRegister-459-thread-1-processing-n:127.0.0.1:38009_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:38009_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:38009/collection1/
   [junit4]   2> 255473 INFO  (coreZkRegister-459-thread-1-processing-n:127.0.0.1:38009_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:38009_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 255473 INFO  (coreZkRegister-459-thread-1-processing-n:127.0.0.1:38009_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:38009_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:38009/collection1/ has no replicas
   [junit4]   2> 255488 INFO  (coreZkRegister-459-thread-1-processing-n:127.0.0.1:38009_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:38009_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 255488 INFO  (OverseerStateUpdate-96248157586980868-127.0.0.1:38009_-n_0000000000) [n:127.0.0.1:38009_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 255525 INFO  (coreZkRegister-459-thread-1-processing-n:127.0.0.1:38009_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:38009_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/control_collection/leaders/shard1/leader after winning as /collections/control_collection/leader_elect/shard1/election/96248157586980868-core_node1-n_0000000000
   [junit4]   2> 255553 INFO  (coreZkRegister-459-thread-1-processing-n:127.0.0.1:38009_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:38009_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:38009/collection1/ shard1
   [junit4]   2> 255565 INFO  (OverseerStateUpdate-96248157586980868-127.0.0.1:38009_-n_0000000000) [n:127.0.0.1:38009_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:38009",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 1
   [junit4]   2> 255687 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 255689 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 255701 INFO  (zkCallback-174-thread-3-processing-n:127.0.0.1:38009_) [n:127.0.0.1:38009_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 255701 INFO  (zkCallback-178-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@50669078 name:ZooKeeperConnection Watcher:127.0.0.1:51810/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 255701 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 255702 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 255702 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 255717 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [    ] o.a.s.c.c.ZkStateReader Loaded empty cluster properties
   [junit4]   2> 255725 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 255727 INFO  (coreZkRegister-459-thread-1-processing-n:127.0.0.1:38009_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:38009_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:38009/collection1/ and leader is http://127.0.0.1:38009/collection1/
   [junit4]   2> 255727 INFO  (coreZkRegister-459-thread-1-processing-n:127.0.0.1:38009_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:38009_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:38009
   [junit4]   2> 255727 INFO  (coreZkRegister-459-thread-1-processing-n:127.0.0.1:38009_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:38009_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 255727 INFO  (coreZkRegister-459-thread-1-processing-n:127.0.0.1:38009_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:38009_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 255727 INFO  (coreZkRegister-459-thread-1-processing-n:127.0.0.1:38009_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:38009_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 255729 INFO  (coreZkRegister-459-thread-1-processing-n:127.0.0.1:38009_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:38009_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Checking legacy cluster state for collection control_collection
   [junit4]   2> 255729 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 255729 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with stateFormat=2
   [junit4]   2> 255729 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 255737 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 255737 INFO  (OverseerStateUpdate-96248157586980868-127.0.0.1:38009_-n_0000000000) [n:127.0.0.1:38009_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:38009",
   [junit4]   2>   "node_name":"127.0.0.1:38009_",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"} current state version: 2
   [junit4]   2> 255737 INFO  (OverseerStateUpdate-96248157586980868-127.0.0.1:38009_-n_0000000000) [n:127.0.0.1:38009_    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:38009",
   [junit4]   2>   "node_name":"127.0.0.1:38009_",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 255754 INFO  (zkCallback-179-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5ce0b8c4 name:ZooKeeperConnection Watcher:127.0.0.1:51810/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 255754 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 255755 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 255798 INFO  (OverseerStateUpdate-96248157586980868-127.0.0.1:38009_-n_0000000000) [n:127.0.0.1:38009_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"create",
   [junit4]   2>   "name":"collection1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "stateFormat":"2"} current state version: 2
   [junit4]   2> 255798 INFO  (OverseerStateUpdate-96248157586980868-127.0.0.1:38009_-n_0000000000) [n:127.0.0.1:38009_    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 255806 INFO  (zkCallback-174-thread-3-processing-n:127.0.0.1:38009_) [n:127.0.0.1:38009_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 255806 INFO  (zkCallback-178-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 255916 INFO  (OverseerStateUpdate-96248157586980868-127.0.0.1:38009_-n_0000000000) [n:127.0.0.1:38009_    ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/collection1/state.json
   [junit4]   2> 255951 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_53C8CD85C44C8CD3-001/shard-1-001/cores/collection1
   [junit4]   2> 255951 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_53C8CD85C44C8CD3-001/shard-1-001
   [junit4]   2> 255954 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [    ] o.e.j.s.Server jetty-9.3.8.v20160314
   [junit4]   2> 255965 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@403633da{/,null,AVAILABLE}
   [junit4]   2> 255966 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [    ] o.e.j.s.ServerConnector Started ServerConnector@6008de6b{HTTP/1.1,[http/1.1]}{127.0.0.1:33672}
   [junit4]   2> 255966 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [    ] o.e.j.s.Server Started @260465ms
   [junit4]   2> 255966 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_53C8CD85C44C8CD3-001/tempDir-001/jetty1, solrconfig=solrconfig.xml, hostContext=/, hostPort=55779, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_53C8CD85C44C8CD3-001/shard-1-001/cores}
   [junit4]   2> 255966 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@73d16e93
   [junit4]   2> 255966 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_53C8CD85C44C8CD3-001/shard-1-001'
   [junit4]   2> 255967 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 255967 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 255967 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 255977 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 256005 INFO  (zkCallback-180-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@548b167d name:ZooKeeperConnection Watcher:127.0.0.1:51810/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 256008 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 256008 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 256034 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 256034 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_53C8CD85C44C8CD3-001/shard-1-001/solr.xml
   [junit4]   2> 256055 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_53C8CD85C44C8CD3-001/shard-1-001/cores
   [junit4]   2> 256061 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [    ] o.a.s.c.CoreContainer New CoreContainer 2044392201
   [junit4]   2> 256061 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[53C8CD85C44C8CD3]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_53C8CD85C44C8CD3-001/shard-1-001]
   [junit4]   2> 256062 WARN  (TEST-LeaderFailoverAfterPa

[...truncated too long message...]

dapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> Caused by: java.lang.NullPointerException
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.blockmanagement.BlocksMap.size(BlocksMap.java:203)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.getTotalBlocks(BlockManager.java:3370)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlocksTotal(FSNamesystem.java:5729)
   [junit4]   2> 	... 54 more
   [junit4]   2> 8341246 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[53C8CD85C44C8CD3]-worker) [    ] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_53C8CD85C44C8CD3-001
   [junit4]   2> Jul 16, 2016 3:07:00 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 35 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=FastCompressingStoredFields(storedFieldsFormat=CompressingStoredFieldsFormat(compressionMode=FAST, chunkSize=28307, maxDocsPerChunk=1, blockSize=5), termVectorsFormat=CompressingTermVectorsFormat(compressionMode=FAST, chunkSize=28307, blockSize=5)), sim=ClassicSimilarity, locale=ar-TN, timezone=Pacific/Funafuti
   [junit4]   2> NOTE: Linux 3.13.0-85-generic amd64/Oracle Corporation 1.8.0_74 (64-bit)/cpus=4,threads=2,free=75667024,total=458752000
   [junit4]   2> NOTE: All tests run in this JVM: [TlogReplayBufferedWhileIndexingTest, TestHashPartitioner, DistributedMLTComponentTest, CustomCollectionTest, AliasIntegrationTest, CdcrReplicationHandlerTest, ConfigSetsAPITest, TestConfigReload, TestExactSharedStatsCache, EnumFieldTest, TestCollationFieldDocValues, TestSolrCoreProperties, SimplePostToolTest, TestRandomCollapseQParserPlugin, TestSizeLimitedDistributedMap, TestSolrConfigHandlerCloud, SpatialFilterTest, TestDynamicLoading, TestQuerySenderListener, OverseerModifyCollectionTest, SpellPossibilityIteratorTest, TestLFUCache, TestSimpleTrackingShardHandler, NotRequiredUniqueKeyTest, TestTrieFacet, TestConfigSetsAPI, TermsComponentTest, SuggestComponentTest, TestRawResponseWriter, MultiTermTest, TestBlendedInfixSuggestions, DistributedFacetPivotSmallAdvancedTest, TestDocTermOrds, HdfsBasicDistributedZk2Test, SSLMigrationTest, TestConfigSetImmutable, TestLeaderInitiatedRecoveryThread, TestBinaryResponseWriter, TestRequestStatusCollectionAPI, TestPKIAuthenticationPlugin, DeleteReplicaTest, TestSolrQueryParserResource, TestNonDefinedSimilarityFactory, TestInitParams, TestFieldCacheSanityChecker, TestUpdate, DirectSolrSpellCheckerTest, HttpPartitionTest, TestSchemaNameResource, DistribCursorPagingTest, DocValuesTest, TestHdfsUpdateLog, TestShardHandlerFactory, MoreLikeThisHandlerTest, HdfsSyncSliceTest, TestRTimerTree, TestSubQueryTransformerCrossCore, SortByFunctionTest, BasicFunctionalityTest, TestFastLRUCache, CoreAdminRequestStatusTest, TestBM25SimilarityFactory, TestCodecSupport, TestManagedResourceStorage, BasicAuthIntegrationTest, TestQueryUtils, HighlighterMaxOffsetTest, TestSolrXml, ChangedSchemaMergeTest, TestManagedSynonymFilterFactory, OverseerTaskQueueTest, TestSortingResponseWriter, ResponseHeaderTest, TimeZoneUtilsTest, ReplicationFactorTest, SolrIndexSplitterTest, OutOfBoxZkACLAndCredentialsProvidersTest, ChaosMonkeySafeLeaderTest, DistributedExpandComponentTest, SolrPluginUtilsTest, TestDynamicFieldCollectionResource, TestFieldCacheVsDocValues, ClusterStateUpdateTest, SharedFSAutoReplicaFailoverTest, ConvertedLegacyTest, TestCloudManagedSchema, CSVRequestHandlerTest, TestSurroundQueryParser, AddBlockUpdateTest, AtomicUpdatesTest, SolrCmdDistributorTest, TestInitQParser, TestMiniSolrCloudClusterSSL, FacetPivotSmallTest, DocumentBuilderTest, TestCustomDocTransformer, SimpleMLTQParserTest, IndexBasedSpellCheckerTest, QueryResultKeyTest, CollectionsAPIDistributedZkTest, TestRandomFaceting, ZkSolrClientTest, ZkCLITest, ShardRoutingCustomTest, TestDistributedSearch, TestCoreContainer, SimpleFacetsTest, StatsComponentTest, SpellCheckComponentTest, TestGroupingSearch, QueryElevationComponentTest, PeerSyncTest, TestFiltering, TestLazyCores, HighlighterTest, SoftAutoCommitTest, ShowFileRequestHandlerTest, TestCoreDiscovery, SignatureUpdateProcessorFactoryTest, TestExtendedDismaxParser, SpellCheckCollatorTest, SuggesterFSTTest, CoreAdminHandlerTest, TestFoldingMultitermQuery, SuggesterTSTTest, SuggesterTest, TestStressLucene, TestTrie, SuggesterWFSTTest, WordBreakSolrSpellCheckerTest, SchemaVersionSpecificBehaviorTest, SolrCoreCheckLockOnStartupTest, FieldMutatingUpdateProcessorTest, TestAtomicUpdateErrorCases, TestWordDelimiterFilterFactory, DirectUpdateHandlerOptimizeTest, DefaultValueUpdateProcessorTest, StatelessScriptUpdateProcessorFactoryTest, TestRemoteStreaming, DistanceFunctionTest, SolrInfoMBeanTest, DisMaxRequestHandlerTest, StandardRequestHandlerTest, TestReversedWildcardFilterFactory, TestQueryTypes, PrimitiveFieldTypeTest, RequestHandlersTest, PathHierarchyTokenizerFactoryTest, TermVectorComponentTest, RequiredFieldsTest, FastVectorHighlighterTest, IndexSchemaRuntimeFieldTest, LoggingHandlerTest, RegexBoostProcessorTest, IndexSchemaTest, ReturnFieldsTest, UpdateRequestProcessorFactoryTest, JSONWriterTest, MBeansHandlerTest, QueryParsingTest, UniqFieldsUpdateProcessorFactoryTest, BinaryUpdateRequestHandlerTest, TestPartialUpdateDeduplication, SearchHandlerTest, HighlighterConfigTest, SOLR749Test, AlternateDirectoryTest, TestQuerySenderNoQuery, ResponseLogComponentTest, SolrIndexConfigTest, BadComponentTest, TestStressRecovery, TestCrossCoreJoin, TestCursorMarkWithoutUniqueKey, TestDistributedMissingSort, TestHighlightDedupGrouping, TestTolerantSearch, TestEmbeddedSolrServerConstructors, TestJettySolrRunner, ActionThrottleTest, AsyncMigrateRouteKeyTest, CloudExitableDirectoryReaderTest, CollectionReloadTest, CollectionsAPIAsyncDistributedZkTest, ConnectionManagerTest, DistributedQueueTest, ForceLeaderTest, TestCloudDeleteByQuery, TestCloudInspectUtil, TestCloudPivotFacet, TestCryptoKeys, TestDownShardTolerantSearch, TestHdfsCloudBackupRestore, TestLocalFSCloudBackupRestore, TestOnReconnectListenerSupport, TestRandomRequestDistribution, TestReplicaProperties, TestShortCircuitedRequests, TestStressLiveNodes, HdfsCollectionsAPIDistributedZkTest]
   [junit4] Completed [621/621 (4!)] on J1 in 3876.76s, 1 test, 1 error <<< FAILURES!

[...truncated 1 lines...]
   [junit4] JVM J1: stdout was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/temp/junit4-J1-20160716_004754_856.sysout
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] java.lang.OutOfMemoryError: Java heap space
   [junit4] Dumping heap to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/heapdumps/java_pid8943.hprof ...
   [junit4] Heap dump file created [620495685 bytes in 6.924 secs]
   [junit4] <<< JVM J1: EOF ----

[...truncated 13 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:770: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:707: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/common-build.xml:536: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/common-build.xml:1443: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/common-build.xml:984: There were test failures: 621 suites (10 ignored), 2642 tests, 2 errors, 2 failures, 46 ignored (31 assumptions) [seed: 53C8CD85C44C8CD3]

Total time: 346 minutes 49 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
No prior successful build to compare, so performing full copy of artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any




[JENKINS] Lucene-Solr-NightlyTests-6.x - Build # 143 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-6.x/143/

5 tests failed.
FAILED:  org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.test

Error Message:
Error from server at http://127.0.0.1:37465/liqc: Error CREATEing SolrCore 'halfcollection_shard1_replica1': Unable to create core [halfcollection_shard1_replica1] Caused by: KeeperErrorCode = Session expired for /overseer/queue/qn-

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:37465/liqc: Error CREATEing SolrCore 'halfcollection_shard1_replica1': Unable to create core [halfcollection_shard1_replica1] Caused by: KeeperErrorCode = Session expired for /overseer/queue/qn-
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:592)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:261)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:250)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.makeRequest(CollectionsAPIDistributedZkTest.java:399)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testErrorHandling(CollectionsAPIDistributedZkTest.java:515)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test(CollectionsAPIDistributedZkTest.java:179)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest

Error Message:
1 thread leaked from SUITE scope at org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest:     1) Thread[id=90374, name=searcherExecutor-9610-thread-1, state=WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest: 
   1) Thread[id=90374, name=searcherExecutor-9610-thread-1, state=WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
	at __randomizedtesting.SeedInfo.seed([681A53DF36E3A80E]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest

Error Message:
There are still zombie threads that couldn't be terminated:    1) Thread[id=90374, name=searcherExecutor-9610-thread-1, state=WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   1) Thread[id=90374, name=searcherExecutor-9610-thread-1, state=WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
	at __randomizedtesting.SeedInfo.seed([681A53DF36E3A80E]:0)


FAILED:  org.apache.solr.cloud.hdfs.HdfsRecoveryZkTest.test

Error Message:
There are still nodes recoverying - waited for 120 seconds

Stack Trace:
java.lang.AssertionError: There are still nodes recoverying - waited for 120 seconds
	at __randomizedtesting.SeedInfo.seed([681A53DF36E3A80E:E04E6C05981FC5F6]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:182)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.waitForRecoveriesToFinish(AbstractFullDistribZkTestBase.java:862)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.waitForThingsToLevelOut(AbstractFullDistribZkTestBase.java:1418)
	at org.apache.solr.cloud.RecoveryZkTest.test(RecoveryZkTest.java:105)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  junit.framework.TestSuite.org.apache.solr.handler.TestReplicationHandler

Error Message:
ObjectTracker found 5 object(s) that were not released!!! [MockDirectoryWrapper, MockDirectoryWrapper, SolrCore, MockDirectoryWrapper, MockDirectoryWrapper]

Stack Trace:
java.lang.AssertionError: ObjectTracker found 5 object(s) that were not released!!! [MockDirectoryWrapper, MockDirectoryWrapper, SolrCore, MockDirectoryWrapper, MockDirectoryWrapper]
	at __randomizedtesting.SeedInfo.seed([681A53DF36E3A80E]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.junit.Assert.assertNull(Assert.java:551)
	at org.apache.solr.SolrTestCaseJ4.teardownTestCases(SolrTestCaseJ4.java:258)
	at sun.reflect.GeneratedMethodAccessor24.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:834)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 12094 lines...]
   [junit4] Suite: org.apache.solr.cloud.hdfs.HdfsRecoveryZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsRecoveryZkTest_681A53DF36E3A80E-001/init-core-data-001
   [junit4]   2> 2393737 INFO  (SUITE-HdfsRecoveryZkTest-seed#[681A53DF36E3A80E]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
   [junit4]   2> 2393740 INFO  (SUITE-HdfsRecoveryZkTest-seed#[681A53DF36E3A80E]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   1> Formatting using clusterid: testClusterID
   [junit4]   2> 2393798 WARN  (SUITE-HdfsRecoveryZkTest-seed#[681A53DF36E3A80E]-worker) [    ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
   [junit4]   2> 2393815 WARN  (SUITE-HdfsRecoveryZkTest-seed#[681A53DF36E3A80E]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 2393817 INFO  (SUITE-HdfsRecoveryZkTest-seed#[681A53DF36E3A80E]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 2393835 INFO  (SUITE-HdfsRecoveryZkTest-seed#[681A53DF36E3A80E]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/hdfs to ./temp/Jetty_localhost_55224_hdfs____6xlj3y/webapp
   [junit4]   2> 2394278 INFO  (SUITE-HdfsRecoveryZkTest-seed#[681A53DF36E3A80E]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:55224
   [junit4]   2> 2394339 WARN  (SUITE-HdfsRecoveryZkTest-seed#[681A53DF36E3A80E]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 2394341 INFO  (SUITE-HdfsRecoveryZkTest-seed#[681A53DF36E3A80E]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 2394356 INFO  (SUITE-HdfsRecoveryZkTest-seed#[681A53DF36E3A80E]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_42371_datanode____7o4cc6/webapp
   [junit4]   2> 2394711 INFO  (SUITE-HdfsRecoveryZkTest-seed#[681A53DF36E3A80E]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:42371
   [junit4]   2> 2394769 WARN  (SUITE-HdfsRecoveryZkTest-seed#[681A53DF36E3A80E]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 2394771 INFO  (SUITE-HdfsRecoveryZkTest-seed#[681A53DF36E3A80E]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 2394788 INFO  (SUITE-HdfsRecoveryZkTest-seed#[681A53DF36E3A80E]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_34818_datanode____.wsrum3/webapp
   [junit4]   2> 2394846 INFO  (IPC Server handler 4 on 38659) [    ] BlockStateChange BLOCK* processReport: from storage DS-5ab92a30-f4bd-46ed-a163-e21db4ab0559 node DatanodeRegistration(127.0.0.1:50919, datanodeUuid=229f1aad-0e95-4d56-b598-f4938e109f2e, infoPort=54636, infoSecurePort=0, ipcPort=60811, storageInfo=lv=-56;cid=testClusterID;nsid=1897704392;c=0), blocks: 0, hasStaleStorage: true, processing time: 0 msecs
   [junit4]   2> 2394846 INFO  (IPC Server handler 4 on 38659) [    ] BlockStateChange BLOCK* processReport: from storage DS-6bc5a271-35ee-4572-9e48-7de81c5a14c6 node DatanodeRegistration(127.0.0.1:50919, datanodeUuid=229f1aad-0e95-4d56-b598-f4938e109f2e, infoPort=54636, infoSecurePort=0, ipcPort=60811, storageInfo=lv=-56;cid=testClusterID;nsid=1897704392;c=0), blocks: 0, hasStaleStorage: false, processing time: 0 msecs
   [junit4]   2> 2395193 INFO  (SUITE-HdfsRecoveryZkTest-seed#[681A53DF36E3A80E]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:34818
   [junit4]   2> 2395367 INFO  (IPC Server handler 4 on 38659) [    ] BlockStateChange BLOCK* processReport: from storage DS-43c33610-91cd-42b3-84fd-70e99edf2410 node DatanodeRegistration(127.0.0.1:41210, datanodeUuid=1029fe4a-1c32-408d-8622-b8d4edc93027, infoPort=43439, infoSecurePort=0, ipcPort=32936, storageInfo=lv=-56;cid=testClusterID;nsid=1897704392;c=0), blocks: 0, hasStaleStorage: true, processing time: 0 msecs
   [junit4]   2> 2395367 INFO  (IPC Server handler 4 on 38659) [    ] BlockStateChange BLOCK* processReport: from storage DS-083325db-aa7d-4d9e-b8ad-be85d3602f2d node DatanodeRegistration(127.0.0.1:41210, datanodeUuid=1029fe4a-1c32-408d-8622-b8d4edc93027, infoPort=43439, infoSecurePort=0, ipcPort=32936, storageInfo=lv=-56;cid=testClusterID;nsid=1897704392;c=0), blocks: 0, hasStaleStorage: false, processing time: 0 msecs
   [junit4]   2> 2395491 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 2395492 INFO  (Thread-6585) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 2395492 INFO  (Thread-6585) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 2395592 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.ZkTestServer start zk server on port:41385
   [junit4]   2> 2395592 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2395594 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2395596 INFO  (zkCallback-1758-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4ff52e0f name:ZooKeeperConnection Watcher:127.0.0.1:41385 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2395596 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2395596 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2395596 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 2395598 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2395606 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2395606 INFO  (zkCallback-1759-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@68402d3a name:ZooKeeperConnection Watcher:127.0.0.1:41385/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2395606 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2395606 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2395606 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 2395608 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 2395609 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 2395610 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 2395611 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 2395611 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 2395613 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
   [junit4]   2> 2395613 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 2395614 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 2395614 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 2395615 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 2395615 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 2395617 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 2395617 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 2395618 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 2395618 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 2395619 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 2395619 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 2395620 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 2395620 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 2395622 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 2395622 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 2395623 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 2395623 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 2395624 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 2395625 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 2395720 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsRecoveryZkTest_681A53DF36E3A80E-001/control-001/cores/collection1
   [junit4]   2> 2395722 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.e.j.s.Server jetty-9.3.8.v20160314
   [junit4]   2> 2395746 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@480c3a48{/,null,AVAILABLE}
   [junit4]   2> 2395746 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.e.j.s.ServerConnector Started ServerConnector@cbb8ac8{HTTP/1.1,[http/1.1]}{127.0.0.1:33221}
   [junit4]   2> 2395746 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.e.j.s.Server Started @2399044ms
   [junit4]   2> 2395746 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=hdfs://localhost:38659/hdfs__localhost_38659__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-6.x_solr_build_solr-core_test_J1_temp_solr.cloud.hdfs.HdfsRecoveryZkTest_681A53DF36E3A80E-001_tempDir-002_control_data, hostContext=/, hostPort=33221, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsRecoveryZkTest_681A53DF36E3A80E-001/control-001/cores}
   [junit4]   2> 2395746 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@73d16e93
   [junit4]   2> 2395747 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsRecoveryZkTest_681A53DF36E3A80E-001/control-001'
   [junit4]   2> 2395747 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 2395747 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 2395747 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2395753 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2395757 INFO  (zkCallback-1760-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@617591a9 name:ZooKeeperConnection Watcher:127.0.0.1:41385/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2395757 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2395757 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2395761 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 2395761 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsRecoveryZkTest_681A53DF36E3A80E-001/control-001/solr.xml
   [junit4]   2> 2395766 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsRecoveryZkTest_681A53DF36E3A80E-001/control-001/cores
   [junit4]   2> 2395766 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.CoreContainer New CoreContainer 2044343108
   [junit4]   2> 2395766 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsRecoveryZkTest_681A53DF36E3A80E-001/control-001]
   [junit4]   2> 2395766 WARN  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsRecoveryZkTest_681A53DF36E3A80E-001/control-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsRecoveryZkTest_681A53DF36E3A80E-001/control-001/lib
   [junit4]   2> 2395766 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,connectionsEvictorSleepDelay : 5000,maxConnectionIdleTime : 40000,
   [junit4]   2> 2395767 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 2395768 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 2395768 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 2395768 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:41385/solr
   [junit4]   2> 2395768 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 2395768 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2395769 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2395769 INFO  (zkCallback-1763-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4e179d34 name:ZooKeeperConnection Watcher:127.0.0.1:41385 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2395770 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2395770 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2395773 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [n:127.0.0.1:33221_    ] o.a.s.c.ZkController Added new OnReconnect listener org.apache.solr.cloud.ZkController$$Lambda$10/96998034@2bd5aa6d
   [junit4]   2> 2395778 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [n:127.0.0.1:33221_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2395778 INFO  (zkCallback-1764-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7421cdbe name:ZooKeeperConnection Watcher:127.0.0.1:41385/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2395778 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [n:127.0.0.1:33221_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2395779 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [n:127.0.0.1:33221_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 2395780 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [n:127.0.0.1:33221_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 2395782 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [n:127.0.0.1:33221_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 2395783 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [n:127.0.0.1:33221_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 2395785 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [n:127.0.0.1:33221_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 2395786 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [n:127.0.0.1:33221_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 2395787 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [n:127.0.0.1:33221_    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 2395788 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [n:127.0.0.1:33221_    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 2395788 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [n:127.0.0.1:33221_    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 2395792 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [n:127.0.0.1:33221_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 2395792 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [n:127.0.0.1:33221_    ] o.a.s.c.c.ZkStateReader Loaded empty cluster properties
   [junit4]   2> 2395793 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [n:127.0.0.1:33221_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (0)
   [junit4]   2> 2395794 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [n:127.0.0.1:33221_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 2395795 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [n:127.0.0.1:33221_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 2395796 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [n:127.0.0.1:33221_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 2395796 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [n:127.0.0.1:33221_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/96398334099259396-127.0.0.1:33221_-n_0000000000
   [junit4]   2> 2395796 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [n:127.0.0.1:33221_    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:33221_
   [junit4]   2> 2395796 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [n:127.0.0.1:33221_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 2395797 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [n:127.0.0.1:33221_    ] o.a.s.c.Overseer Overseer (id=96398334099259396-127.0.0.1:33221_-n_0000000000) starting
   [junit4]   2> 2395798 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [n:127.0.0.1:33221_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 2395802 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [n:127.0.0.1:33221_    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 2395805 INFO  (OverseerStateUpdate-96398334099259396-127.0.0.1:33221_-n_0000000000) [n:127.0.0.1:33221_    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 2395806 INFO  (OverseerStateUpdate-96398334099259396-127.0.0.1:33221_-n_0000000000) [n:127.0.0.1:33221_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (0)
   [junit4]   2> 2395809 INFO  (OverseerCollectionConfigSetProcessor-96398334099259396-127.0.0.1:33221_-n_0000000000) [n:127.0.0.1:33221_    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 2395812 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [n:127.0.0.1:33221_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:33221_
   [junit4]   2> 2395812 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [n:127.0.0.1:33221_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:33221_
   [junit4]   2> 2395813 INFO  (zkCallback-1764-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [0])
   [junit4]   2> 2395817 INFO  (zkCallback-1764-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 2395831 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [n:127.0.0.1:33221_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 2395831 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [n:127.0.0.1:33221_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 2395833 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [n:127.0.0.1:33221_    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsRecoveryZkTest_681A53DF36E3A80E-001/control-001/cores
   [junit4]   2> 2395834 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [n:127.0.0.1:33221_    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, loadOnStartup=true, configSetProperties=configsetprops.json, collection=control_collection, coreNodeName=, dataDir=data/, shard=}
   [junit4]   2> 2395834 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [n:127.0.0.1:33221_    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsRecoveryZkTest_681A53DF36E3A80E-001/control-001/cores/collection1
   [junit4]   2> 2395834 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [n:127.0.0.1:33221_    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 2395846 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 2395846 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 2395847 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection   x:collection1] o.a.s.c.ZkController Collection control_collection not visible yet, but flagging it so a watch is registered when it becomes visible
   [junit4]   2> 2395848 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Deleting data for [control_collection]
   [junit4]   2> 2395856 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [n:127.0.0.1:33221_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1
   [junit4]   2> 2395856 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [n:127.0.0.1:33221_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 2395856 INFO  (OverseerStateUpdate-96398334099259396-127.0.0.1:33221_-n_0000000000) [n:127.0.0.1:33221_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:33221",
   [junit4]   2>   "node_name":"127.0.0.1:33221_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 2395856 INFO  (OverseerStateUpdate-96398334099259396-127.0.0.1:33221_-n_0000000000) [n:127.0.0.1:33221_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:33221",
   [junit4]   2>   "node_name":"127.0.0.1:33221_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 2395857 INFO  (OverseerStateUpdate-96398334099259396-127.0.0.1:33221_-n_0000000000) [n:127.0.0.1:33221_    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 2395857 INFO  (OverseerStateUpdate-96398334099259396-127.0.0.1:33221_-n_0000000000) [n:127.0.0.1:33221_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 2395858 INFO  (zkCallback-1764-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 2395875 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 2395875 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 2395875 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 2395875 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 2395875 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/control_collection]
   [junit4]   2> 2395876 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/control_collection] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 2395876 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsRecoveryZkTest_681A53DF36E3A80E-001/control-001/cores/collection1'
   [junit4]   2> 2395876 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 2395876 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 2395877 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 2395890 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 2395893 WARN  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection   x:collection1] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
   [junit4]   2> 2395894 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.2.0
   [junit4]   2> 2395903 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 2395908 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 2396039 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 2396040 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 2396040 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 2396043 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 2396077 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1
   [junit4]   2> 2396077 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 2396077 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.HdfsDirectoryFactory
   [junit4]   2> 2396078 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:38659/solr_hdfs_home
   [junit4]   2> 2396078 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
   [junit4]   2> 2396078 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsRecoveryZkTest_681A53DF36E3A80E-001/control-001/cores/collection1], dataDir=[hdfs://localhost:38659/solr_hdfs_home/control_collection/core_node1/data/]
   [junit4]   2> 2396078 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@259a0e9d
   [junit4]   2> 2396078 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:38659/solr_hdfs_home/control_collection/core_node1/data/snapshot_metadata
   [junit4]   2> 2396089 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
   [junit4]   2> 2396089 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [16777216] will allocate [1] slabs and use ~[16777216] bytes
   [junit4]   2> 2396089 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
   [junit4]   2> 2396106 WARN  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.h.HdfsDirectory The NameNode is in SafeMode - Solr will wait 5 seconds and try again.
   [junit4]   2> 2401111 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
   [junit4]   2> 2401111 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:38659/solr_hdfs_home/control_collection/core_node1/data/snapshot_metadata
   [junit4]   2> 2401111 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.s.SolrSnapshotMetaDataManager Loading from snapshot metadata file...
   [junit4]   2> 2401112 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:38659/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 2401121 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:38659/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 2401123 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=hdfs://localhost:38659/solr_hdfs_home/control_collection/core_node1/data/index/
   [junit4]   2> 2401141 WARN  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory 'hdfs:/localhost:38659/solr_hdfs_home/control_collection/core_node1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 2401141 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:38659/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 2401154 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
   [junit4]   2> 2401155 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [16777216] will allocate [1] slabs and use ~[16777216] bytes
   [junit4]   2> 2401155 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
   [junit4]   2> 2401186 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
   [junit4]   2> 2401186 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:38659/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 2401186 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=15, maxMergeAtOnceExplicit=12, maxMergedSegmentMB=36.7431640625, floorSegmentMB=1.541015625, forceMergeDeletesPctAllowed=6.960733237335141, segmentsPerTier=42.0, maxCFSSegmentSizeMB=1.6484375, noCFSRatio=0.26891872459162586
   [junit4]   2> 2401271 INFO  (IPC Server handler 6 on 38659) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:41210 is added to blk_1073741825_1001{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-6bc5a271-35ee-4572-9e48-7de81c5a14c6:NORMAL:127.0.0.1:50919|RBW], ReplicaUC[[DISK]DS-43c33610-91cd-42b3-84fd-70e99edf2410:NORMAL:127.0.0.1:41210|FINALIZED]]} size 0
   [junit4]   2> 2401273 INFO  (IPC Server handler 7 on 38659) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:50919 is added to blk_1073741825_1001 size 71
   [junit4]   2> 2401274 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(BlockDirectory(HdfsDirectory@hdfs://localhost:38659/solr_hdfs_home/control_collection/core_node1/data/index lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@7b387472); maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 2401274 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 2401280 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in hdfs://localhost:38659/solr_hdfs_home/control_collection/core_node1/data/
   [junit4]   2> 2401280 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 2401280 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 2401280 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 2401280 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 2401280 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 2401280 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 2401281 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 2401281 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 2401281 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 2401281 WARN  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
   [junit4]   2> 2401282 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2401283 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2401284 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2401285 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2401288 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/mbeans,standard,/update/json/docs,/admin/luke,/export,/get,/admin/properties,/update/json,/admin/threads,/dump,/update/csv,/sql,/graph,/admin/segments,/admin/system,/replication,/config,/stream,/schema,/admin/plugins,/admin/logging,/admin/ping,/update,/admin/file,/terms
   [junit4]   2> 2401288 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 2401290 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
   [junit4]   2> 2401290 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 2401290 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=2
   [junit4]   2> 2401317 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 2401317 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 2401319 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=76782694, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
   [junit4]   2> 2401343 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(BlockDirectory(HdfsDirectory@hdfs://localhost:38659/solr_hdfs_home/control_collection/core_node1/data/index lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@7b387472); maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 2401343 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 2401343 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@7bd1df9[collection1] main]
   [junit4]   2> 2401343 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/control_collection]
   [junit4]   2> 2401344 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/control_collection] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 2401344 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 2401344 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 2401344 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 2401344 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 2401345 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 2401345 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 2401345 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 2401345 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 2401346 INFO  (searcherExecutor-3732-thread-1-processing-n:127.0.0.1:33221_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@7bd1df9[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2401346 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 2401346 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 2401346 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 2401346 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1542373351725465600
   [junit4]   2> 2401348 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 1.0ms to seed version buckets with highest version 1542373351725465600
   [junit4]   2> 2401348 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 2401348 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 2401349 INFO  (coreZkRegister-3724-thread-1-processing-n:127.0.0.1:33221_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:33221 collection:control_collection shard:shard1
   [junit4]   2> 2401350 INFO  (coreZkRegister-3724-thread-1-processing-n:127.0.0.1:33221_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 2401353 INFO  (coreZkRegister-3724-thread-1-processing-n:127.0.0.1:33221_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/96398334099259396-core_node1-n_0000000000
   [junit4]   2> 2401353 INFO  (coreZkRegister-3724-thread-1-processing-n:127.0.0.1:33221_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard=shard1 and weAreReplacement=false and leaderVoteWait=10000
   [junit4]   2> 2401355 INFO  (coreZkRegister-3724-thread-1-processing-n:127.0.0.1:33221_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 2401355 INFO  (coreZkRegister-3724-thread-1-processing-n:127.0.0.1:33221_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 2401355 INFO  (coreZkRegister-3724-thread-1-processing-n:127.0.0.1:33221_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:33221/collection1/
   [junit4]   2> 2401355 INFO  (coreZkRegister-3724-thread-1-processing-n:127.0.0.1:33221_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 2401355 INFO  (coreZkRegister-3724-thread-1-processing-n:127.0.0.1:33221_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:33221/collection1/ has no replicas
   [junit4]   2> 2401355 INFO  (OverseerStateUpdate-96398334099259396-127.0.0.1:33221_-n_0000000000) [n:127.0.0.1:33221_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 2401356 INFO  (coreZkRegister-3724-thread-1-processing-n:127.0.0.1:33221_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 2401358 INFO  (coreZkRegister-3724-thread-1-processing-n:127.0.0.1:33221_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/control_collection/leaders/shard1/leader after winning as /collections/control_collection/leader_elect/shard1/election/96398334099259396-core_node1-n_0000000000
   [junit4]   2> 2401359 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2401359 INFO  (coreZkRegister-3724-thread-1-processing-n:127.0.0.1:33221_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:33221/collection1/ shard1
   [junit4]   2> 2401377 INFO  (OverseerStateUpdate-96398334099259396-127.0.0.1:33221_-n_0000000000) [n:127.0.0.1:33221_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:33221",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 2
   [junit4]   2> 2401385 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2401389 INFO  (zkCallback-1764-thread-4-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 2401404 INFO  (zkCallback-1768-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@59e9028a name:ZooKeeperConnection Watcher:127.0.0.1:41385/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2401412 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2401412 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2401412 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 2401414 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.c.ZkStateReader Loaded empty cluster properties
   [junit4]   2> 2401414 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 2401415 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 2401479 INFO  (zkCallback-1768-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 2401480 INFO  (zkCallback-1764-thread-4-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 2401512 INFO  (coreZkRegister-3724-thread-1-processing-n:127.0.0.1:33221_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:33221/collection1/ and leader is http://127.0.0.1:33221/collection1/
   [junit4]   2> 2401512 INFO  (coreZkRegister-3724-thread-1-processing-n:127.0.0.1:33221_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:33221
   [junit4]   2> 2401512 INFO  (coreZkRegister-3724-thread-1-processing-n:127.0.0.1:33221_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 2401512 INFO  (coreZkRegister-3724-thread-1-processing-n:127.0.0.1:33221_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 2401512 INFO  (coreZkRegister-3724-thread-1-processing-n:127.0.0.1:33221_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 2401513 INFO  (coreZkRegister-3724-thread-1-processing-n:127.0.0.1:33221_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Checking legacy cluster state for collection control_collection
   [junit4]   2> 2401514 INFO  (OverseerStateUpdate-96398334099259396-127.0.0.1:33221_-n_0000000000) [n:127.0.0.1:33221_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "dataDir":"hdfs://localhost:38659/solr_hdfs_home/control_collection/core_node1/data/",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:33221",
   [junit4]   2>   "node_name":"127.0.0.1:33221_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "ulogDir":"hdfs://localhost:38659/solr_hdfs_home/control_collection/core_node1/data/tlog"} current state version: 3
   [junit4]   2> 2401514 INFO  (OverseerStateUpdate-96398334099259396-127.0.0.1:33221_-n_0000000000) [n:127.0.0.1:33221_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "dataDir":"hdfs://localhost:38659/solr_hdfs_home/control_collection/core_node1/data/",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:33221",
   [junit4]   2>   "node_name":"127.0.0.1:33221_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "ulogDir":"hdfs://localhost:38659/solr_hdfs_home/control_collection/core_node1/data/tlog"}
   [junit4]   2> 2401519 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsRecoveryZkTest_681A53DF36E3A80E-001/shard-1-001/cores/collection1
   [junit4]   2> 2401520 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsRecoveryZkTest_681A53DF36E3A80E-001/shard-1-001
   [junit4]   2> 2401520 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.e.j.s.Server jetty-9.3.8.v20160314
   [junit4]   2> 2401522 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@4151ca52{/,null,AVAILABLE}
   [junit4]   2> 2401522 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.e.j.s.ServerConnector Started ServerConnector@74f34a24{HTTP/1.1,[http/1.1]}{127.0.0.1:50634}
   [junit4]   2> 2401522 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.e.j.s.Server Started @2404820ms
   [junit4]   2> 2401522 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=hdfs://localhost:38659/hdfs__localhost_38659__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-6.x_solr_build_solr-core_test_J1_temp_solr.cloud.hdfs.HdfsRecoveryZkTest_681A53DF36E3A80E-001_tempDir-002_jetty1, solrconfig=solrconfig.xml, hostContext=/, hostPort=50634, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsRecoveryZkTest_681A53DF36E3A80E-001/shard-1-001/cores}
   [junit4]   2> 2401522 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@73d16e93
   [junit4]   2> 2401522 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsRecoveryZkTest_681A53DF36E3A80E-001/shard-1-001'
   [junit4]   2> 2401523 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 2401523 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 2401523 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2401524 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2401524 INFO  (zkCallback-1769-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3758f651 name:ZooKeeperConnection Watcher:127.0.0.1:41385/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2401524 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2401524 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2401525 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 2401525 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsRecoveryZkTest_681A53DF36E3A80E-001/shard-1-001/solr.xml
   [junit4]   2> 2401530 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsRecoveryZkTest_681A53DF36E3A80E-001/shard-1-001/cores
   [junit4]   2> 2401531 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.CoreContainer New CoreContainer 1839996372
   [junit4]   2> 2401531 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jen

[...truncated too long message...]

dPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]   2>         at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> Aug 11, 2016 2:08:08 PM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
   [junit4]   2> INFO: Starting to interrupt leaked threads:
   [junit4]   2>    1) Thread[id=90374, name=searcherExecutor-9610-thread-1, state=WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]
   [junit4]   2> Aug 11, 2016 2:08:11 PM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
   [junit4]   2> SEVERE: There are still zombie threads that couldn't be terminated:
   [junit4]   2>    1) Thread[id=90374, name=searcherExecutor-9610-thread-1, state=WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]
   [junit4]   2>         at sun.misc.Unsafe.park(Native Method)
   [junit4]   2>         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
   [junit4]   2>         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
   [junit4]   2>         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]   2>         at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene62): {other_tl1=PostingsFormat(name=LuceneVarGapFixedInterval), range_facet_l_dv=Lucene50(blocksize=128), _version_=PostingsFormat(name=LuceneVarGapFixedInterval), multiDefault=PostingsFormat(name=Memory doPackFST= false), a_t=PostingsFormat(name=LuceneVarGapFixedInterval), intDefault=PostingsFormat(name=LuceneVarGapFixedInterval), a_i1=PostingsFormat(name=LuceneVarGapFixedInterval), id=Lucene50(blocksize=128), range_facet_i_dv=PostingsFormat(name=LuceneVarGapFixedInterval), text=PostingsFormat(name=Asserting), range_facet_l=PostingsFormat(name=LuceneVarGapFixedInterval), timestamp=PostingsFormat(name=LuceneVarGapFixedInterval)}, docValues:{range_facet_l_dv=DocValuesFormat(name=Asserting), range_facet_i_dv=DocValuesFormat(name=Lucene54), timestamp=DocValuesFormat(name=Lucene54)}, maxPointsInLeafNode=1068, maxMBSortInHeap=5.338083296791959, sim=ClassicSimilarity, locale=cs, timezone=Antarctica/Casey
   [junit4]   2> NOTE: Linux 3.13.0-85-generic amd64/Oracle Corporation 1.8.0_74 (64-bit)/cpus=4,threads=3,free=87685600,total=467140608
   [junit4]   2> NOTE: All tests run in this JVM: [TestHighlightDedupGrouping, OverseerTaskQueueTest, TestSolr4Spatial, TestNumericTerms32, CacheHeaderTest, BasicZkTest, FileBasedSpellCheckerTest, TestComponentsName, TestRandomFlRTGCloud, TestStressLucene, TestConfigOverlay, SimpleMLTQParserTest, TestLeaderElectionZkExpiry, CdcrRequestHandlerTest, SolrRequestParserTest, TestCloudInspectUtil, TestSolrDynamicMBean, TestLockTree, DocExpirationUpdateProcessorFactoryTest, TestLMDirichletSimilarityFactory, DeleteInactiveReplicaTest, RAMDirectoryFactoryTest, DisMaxRequestHandlerTest, TestCustomSort, TestDocSet, TestDownShardTolerantSearch, TestConfig, EchoParamsTest, ResourceLoaderTest, StandardRequestHandlerTest, PreAnalyzedFieldTest, BufferStoreTest, TlogReplayBufferedWhileIndexingTest, DeleteReplicaTest, SpatialHeatmapFacetsTest, OverseerModifyCollectionTest, JsonLoaderTest, TestRandomFaceting, TestValueSourceCache, UUIDUpdateProcessorFallbackTest, PolyFieldTest, DistributedQueueTest, HdfsChaosMonkeySafeLeaderTest, ActionThrottleTest, TestReloadDeadlock, TestHighFrequencyDictionaryFactory, TestLazyCores, QueryEqualityTest, TestManagedSchema, TestGeoJSONResponseWriter, DistributedSpellCheckComponentTest, PreAnalyzedUpdateProcessorTest, TestRestoreCore, SyncSliceTest, DistributedQueryElevationComponentTest, PathHierarchyTokenizerFactoryTest, LeaderInitiatedRecoveryOnShardRestartTest, TestUniqueKeyFieldResource, BJQParserTest, TestChildDocTransformer, TestPseudoReturnFields, JSONWriterTest, URLClassifyProcessorTest, TestMinMaxOnMultiValuedField, TestFieldCacheVsDocValues, TestNonDefinedSimilarityFactory, ExternalFileFieldSortTest, BasicDistributedZkTest, TestExactSharedStatsCache, OutputWriterTest, SolrXmlInZkTest, BigEndianAscendingWordDeserializerTest, CursorPagingTest, CleanupOldIndexTest, OutOfBoxZkACLAndCredentialsProvidersTest, TestCrossCoreJoin, TestBlendedInfixSuggestions, SegmentsInfoRequestHandlerTest, TestPKIAuthenticationPlugin, TestReloadAndDeleteDocs, BasicAuthIntegrationTest, SynonymTokenizerTest, TestCollectionAPI, TestSmileRequest, ReplaceNodeTest, CopyFieldTest, SignatureUpdateProcessorFactoryTest, DistribCursorPagingTest, HttpPartitionTest, TestArbitraryIndexDir, HdfsDirectoryFactoryTest, TestSubQueryTransformerDistrib, TestFieldTypeCollectionResource, QueryElevationComponentTest, TestIntervalFaceting, TestCopyFieldCollectionResource, BlockJoinFacetSimpleTest, TestFieldCacheSanityChecker, TestDistributedSearch, ConnectionManagerTest, TestReplicationHandler, CollectionTooManyReplicasTest, LeaderFailoverAfterPartitionTest, OverseerCollectionConfigSetProcessorTest, OverseerStatusTest, RemoteQueryErrorTest, RollingRestartTest, SharedFSAutoReplicaFailoverUtilsTest, SimpleCollectionCreateDeleteTest, SolrCloudExampleTest, TestConfigSetsAPI, TestConfigSetsAPIExclusivity, TestConfigSetsAPIZkFailure, TestDistribDocBasedVersion, TestMiniSolrCloudClusterBase, TestMiniSolrCloudClusterSSL, TestOnReconnectListenerSupport, TestRebalanceLeaders, TestRequestForwarding, TestSSLRandomization, TestShortCircuitedRequests, TestSolrCloudWithDelegationTokens, TestStressCloudBlindAtomicUpdates, HdfsCollectionsAPIDistributedZkTest]
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=HdfsCollectionsAPIDistributedZkTest -Dtests.seed=681A53DF36E3A80E -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=cs -Dtests.timezone=Antarctica/Casey -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
   [junit4] ERROR   0.00s J0 | HdfsCollectionsAPIDistributedZkTest (suite) <<<
   [junit4]    > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest: 
   [junit4]    >    1) Thread[id=90374, name=searcherExecutor-9610-thread-1, state=WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
   [junit4]    >         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([681A53DF36E3A80E]:0)Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   [junit4]    >    1) Thread[id=90374, name=searcherExecutor-9610-thread-1, state=WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
   [junit4]    >         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([681A53DF36E3A80E]:0)
   [junit4] Completed [501/629 (3!)] on J0 in 654.30s, 1 test, 3 errors <<< FAILURES!

[...truncated 378 lines...]
   [junit4] JVM J0: stdout was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/temp/junit4-J0-20160811_124407_149.sysout
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] java.lang.OutOfMemoryError: GC overhead limit exceeded
   [junit4] Dumping heap to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/heapdumps/java_pid30416.hprof ...
   [junit4] Heap dump file created [617214879 bytes in 12.413 secs]
   [junit4] <<< JVM J0: EOF ----

[...truncated 11055 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:771: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:715: Some of the tests produced a heap dump, but did not fail. Maybe a suppressed OutOfMemoryError? Dumps created:
* java_pid30416.hprof

Total time: 272 minutes 20 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any




[JENKINS] Lucene-Solr-NightlyTests-6.x - Build # 142 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-6.x/142/

1 tests failed.
FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=134092, name=collection1, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=134092, name=collection1, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:34073: collection already exists: awholynewstresscollection_collection1_3
	at __randomizedtesting.SeedInfo.seed([A64F024E870E254C]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:592)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:261)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:250)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:403)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:356)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1291)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1061)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:997)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1599)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1620)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:987)




Build Log:
[...truncated 11969 lines...]
   [junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A64F024E870E254C-001/init-core-data-001
   [junit4]   2> 2249274 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[A64F024E870E254C]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
   [junit4]   2> 2249275 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[A64F024E870E254C]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   2> 2249290 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 2249293 INFO  (Thread-12227) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 2249293 INFO  (Thread-12227) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 2249393 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.ZkTestServer start zk server on port:37694
   [junit4]   2> 2249393 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2249393 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2249395 INFO  (zkCallback-32724-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@58994e50 name:ZooKeeperConnection Watcher:127.0.0.1:37694 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2249395 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2249395 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2249395 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 2249397 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2249397 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x15674ff350f0000, likely client has closed socket
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 2249400 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2249401 INFO  (zkCallback-32725-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@796e4a4 name:ZooKeeperConnection Watcher:127.0.0.1:37694/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2249401 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2249401 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2249401 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 2249403 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 2249404 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 2249405 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 2249406 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 2249406 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 2249426 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 2249426 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 2249427 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 2249427 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 2249428 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 2249428 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 2249429 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 2249430 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 2249434 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 2249435 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 2249435 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 2249436 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 2249437 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 2249437 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 2249438 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 2249438 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 2249439 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 2249439 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 2249440 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 2249440 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 2249441 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2249461 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2249473 INFO  (zkCallback-32727-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1a1c7b57 name:ZooKeeperConnection Watcher:127.0.0.1:37694/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2249474 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2249474 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2249613 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A64F024E870E254C-001/control-001/cores/collection1
   [junit4]   2> 2249615 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.e.j.s.Server jetty-9.3.8.v20160314
   [junit4]   2> 2249722 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@48d51d6f{/,null,AVAILABLE}
   [junit4]   2> 2249724 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.e.j.s.ServerConnector Started ServerConnector@2689aa7{SSL,[ssl, http/1.1]}{127.0.0.1:34073}
   [junit4]   2> 2249724 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.e.j.s.Server Started @2252411ms
   [junit4]   2> 2249724 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A64F024E870E254C-001/tempDir-001/control/data, hostContext=/, hostPort=34073, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A64F024E870E254C-001/control-001/cores}
   [junit4]   2> 2249724 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@73d16e93
   [junit4]   2> 2249724 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A64F024E870E254C-001/control-001'
   [junit4]   2> 2249724 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 2249724 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 2249725 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2249743 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2249773 INFO  (zkCallback-32728-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3e97244a name:ZooKeeperConnection Watcher:127.0.0.1:37694/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2249773 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2249774 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2249774 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 2249774 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A64F024E870E254C-001/control-001/solr.xml
   [junit4]   2> 2249779 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A64F024E870E254C-001/control-001/cores
   [junit4]   2> 2249779 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.CoreContainer New CoreContainer 916234640
   [junit4]   2> 2249779 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A64F024E870E254C-001/control-001]
   [junit4]   2> 2249779 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A64F024E870E254C-001/control-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A64F024E870E254C-001/control-001/lib
   [junit4]   2> 2249779 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,connectionsEvictorSleepDelay : 5000,maxConnectionIdleTime : 40000,
   [junit4]   2> 2249780 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 2249781 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 2249781 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 2249781 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:37694/solr
   [junit4]   2> 2249781 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 2249781 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2249794 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2249809 INFO  (zkCallback-32731-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@310de681 name:ZooKeeperConnection Watcher:127.0.0.1:37694 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2249810 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2249810 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2249811 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [n:127.0.0.1:34073_    ] o.a.s.c.ZkController Added new OnReconnect listener org.apache.solr.cloud.ZkController$$Lambda$12/2101902147@32b69bee
   [junit4]   2> 2249833 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [n:127.0.0.1:34073_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2249849 INFO  (zkCallback-32732-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7180f7c6 name:ZooKeeperConnection Watcher:127.0.0.1:37694/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2249850 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [n:127.0.0.1:34073_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2249850 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [n:127.0.0.1:34073_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 2249852 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [n:127.0.0.1:34073_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 2249853 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [n:127.0.0.1:34073_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 2249855 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [n:127.0.0.1:34073_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 2249856 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [n:127.0.0.1:34073_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 2249857 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [n:127.0.0.1:34073_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 2249858 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [n:127.0.0.1:34073_    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 2249858 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [n:127.0.0.1:34073_    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 2249859 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [n:127.0.0.1:34073_    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 2249860 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [n:127.0.0.1:34073_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 2249860 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [n:127.0.0.1:34073_    ] o.a.s.c.c.ZkStateReader Loaded cluster properties: {urlScheme=https}
   [junit4]   2> 2249860 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [n:127.0.0.1:34073_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (0)
   [junit4]   2> 2249862 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [n:127.0.0.1:34073_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 2249863 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [n:127.0.0.1:34073_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 2249863 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [n:127.0.0.1:34073_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 2249864 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [n:127.0.0.1:34073_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/96393081490702341-127.0.0.1:34073_-n_0000000000
   [junit4]   2> 2249864 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [n:127.0.0.1:34073_    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:34073_
   [junit4]   2> 2249864 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [n:127.0.0.1:34073_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 2249865 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [n:127.0.0.1:34073_    ] o.a.s.c.Overseer Overseer (id=96393081490702341-127.0.0.1:34073_-n_0000000000) starting
   [junit4]   2> 2249866 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [n:127.0.0.1:34073_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 2249869 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [n:127.0.0.1:34073_    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 2249881 INFO  (OverseerCollectionConfigSetProcessor-96393081490702341-127.0.0.1:34073_-n_0000000000) [n:127.0.0.1:34073_    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 2249881 INFO  (OverseerStateUpdate-96393081490702341-127.0.0.1:34073_-n_0000000000) [n:127.0.0.1:34073_    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 2249895 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [n:127.0.0.1:34073_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:34073_
   [junit4]   2> 2249895 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [n:127.0.0.1:34073_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:34073_
   [junit4]   2> 2249895 INFO  (OverseerStateUpdate-96393081490702341-127.0.0.1:34073_-n_0000000000) [n:127.0.0.1:34073_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (0)
   [junit4]   2> 2249896 INFO  (zkCallback-32732-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [0])
   [junit4]   2> 2249897 INFO  (zkCallback-32732-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 2249909 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [n:127.0.0.1:34073_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 2249910 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [n:127.0.0.1:34073_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 2249910 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [n:127.0.0.1:34073_    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A64F024E870E254C-001/control-001/cores
   [junit4]   2> 2249911 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [n:127.0.0.1:34073_    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, loadOnStartup=true, configSetProperties=configsetprops.json, collection=control_collection, coreNodeName=, dataDir=data/, shard=}
   [junit4]   2> 2249911 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [n:127.0.0.1:34073_    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A64F024E870E254C-001/control-001/cores/collection1
   [junit4]   2> 2249911 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [n:127.0.0.1:34073_    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 2249912 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 2249912 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 2249912 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [n:127.0.0.1:34073_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1
   [junit4]   2> 2249912 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [n:127.0.0.1:34073_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 2249913 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection   x:collection1] o.a.s.c.ZkController Collection control_collection not visible yet, but flagging it so a watch is registered when it becomes visible
   [junit4]   2> 2249913 INFO  (OverseerStateUpdate-96393081490702341-127.0.0.1:34073_-n_0000000000) [n:127.0.0.1:34073_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:34073",
   [junit4]   2>   "node_name":"127.0.0.1:34073_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 2249914 INFO  (OverseerStateUpdate-96393081490702341-127.0.0.1:34073_-n_0000000000) [n:127.0.0.1:34073_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:34073",
   [junit4]   2>   "node_name":"127.0.0.1:34073_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 2249914 INFO  (OverseerStateUpdate-96393081490702341-127.0.0.1:34073_-n_0000000000) [n:127.0.0.1:34073_    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 2249914 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Deleting data for [control_collection]
   [junit4]   2> 2249914 INFO  (OverseerStateUpdate-96393081490702341-127.0.0.1:34073_-n_0000000000) [n:127.0.0.1:34073_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 2249915 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 2249915 INFO  (zkCallback-32732-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 2250916 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 2250916 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 2250916 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 2250916 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/control_collection]
   [junit4]   2> 2250917 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/control_collection] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 2250917 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A64F024E870E254C-001/control-001/cores/collection1'
   [junit4]   2> 2250917 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 2250917 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 2250918 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 2250922 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 2250925 WARN  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection   x:collection1] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
   [junit4]   2> 2250926 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.2.0
   [junit4]   2> 2250980 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 2250985 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 2251087 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 2251093 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 2251094 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 2251101 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 2251103 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 2251104 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 2251104 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 2251106 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1
   [junit4]   2> 2251106 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 2251106 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 2251106 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A64F024E870E254C-001/control-001/cores/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A64F024E870E254C-001/control-001/cores/collection1/data/]
   [junit4]   2> 2251106 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@2c21dbd4
   [junit4]   2> 2251107 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A64F024E870E254C-001/control-001/cores/collection1/data/snapshot_metadata
   [junit4]   2> 2251107 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.s.SolrSnapshotMetaDataManager Loading from snapshot metadata file...
   [junit4]   2> 2251107 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A64F024E870E254C-001/control-001/cores/collection1/data
   [junit4]   2> 2251107 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A64F024E870E254C-001/control-001/cores/collection1/data/index/
   [junit4]   2> 2251107 WARN  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A64F024E870E254C-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 2251107 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A64F024E870E254C-001/control-001/cores/collection1/data/index
   [junit4]   2> 2251107 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=16, maxMergeAtOnceExplicit=38, maxMergedSegmentMB=0.7978515625, floorSegmentMB=0.5205078125, forceMergeDeletesPctAllowed=17.571953688359336, segmentsPerTier=14.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.8681645510787229
   [junit4]   2> 2251108 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=RawDirectoryWrapper(RAMDirectory@15174f87 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@7fbb82bb),segFN=segments_1,generation=1}
   [junit4]   2> 2251108 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 2251141 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 2251141 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 2251142 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 2251142 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 2251142 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 2251142 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 2251149 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 2251150 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 2251150 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 2251150 WARN  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
   [junit4]   2> 2251152 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2251152 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2251153 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2251154 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2251156 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/mbeans,standard,/update/json/docs,/admin/luke,/export,/get,/admin/properties,/update/json,/admin/threads,/dump,/update/csv,/sql,/graph,/admin/segments,/admin/system,/replication,/config,/stream,/schema,/admin/plugins,/admin/logging,/admin/ping,/update,/admin/file,/terms
   [junit4]   2> 2251156 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 2251157 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A64F024E870E254C-001/control-001/cores/collection1/data/
   [junit4]   2> 2251157 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A64F024E870E254C-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 2251167 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 2251167 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 2251168 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 2251168 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 2251168 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=10, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
   [junit4]   2> 2251169 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=RawDirectoryWrapper(RAMDirectory@15174f87 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@7fbb82bb),segFN=segments_1,generation=1}
   [junit4]   2> 2251169 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 2251169 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@48016e23[collection1] main]
   [junit4]   2> 2251169 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/control_collection]
   [junit4]   2> 2251174 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/control_collection] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 2251174 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 2251174 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 2251174 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 2251174 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 2251175 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 2251175 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 2251175 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 2251175 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 2251176 INFO  (searcherExecutor-10491-thread-1-processing-n:127.0.0.1:34073_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@48016e23[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2251176 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 2251176 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 2251176 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 2251176 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1542289305824657408
   [junit4]   2> 2251178 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 1.0ms to seed version buckets with highest version 1542289305824657408
   [junit4]   2> 2251178 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 2251178 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 2251179 INFO  (coreZkRegister-10484-thread-1-processing-n:127.0.0.1:34073_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:https://127.0.0.1:34073 collection:control_collection shard:shard1
   [junit4]   2> 2251180 INFO  (coreZkRegister-10484-thread-1-processing-n:127.0.0.1:34073_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 2251182 INFO  (coreZkRegister-10484-thread-1-processing-n:127.0.0.1:34073_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/96393081490702341-core_node1-n_0000000000
   [junit4]   2> 2251182 INFO  (coreZkRegister-10484-thread-1-processing-n:127.0.0.1:34073_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard=shard1 and weAreReplacement=false and leaderVoteWait=10000
   [junit4]   2> 2251184 INFO  (coreZkRegister-10484-thread-1-processing-n:127.0.0.1:34073_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 2251184 INFO  (coreZkRegister-10484-thread-1-processing-n:127.0.0.1:34073_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 2251184 INFO  (coreZkRegister-10484-thread-1-processing-n:127.0.0.1:34073_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:34073/collection1/
   [junit4]   2> 2251184 INFO  (coreZkRegister-10484-thread-1-processing-n:127.0.0.1:34073_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 2251184 INFO  (coreZkRegister-10484-thread-1-processing-n:127.0.0.1:34073_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy https://127.0.0.1:34073/collection1/ has no replicas
   [junit4]   2> 2251184 INFO  (OverseerStateUpdate-96393081490702341-127.0.0.1:34073_-n_0000000000) [n:127.0.0.1:34073_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 2251185 INFO  (coreZkRegister-10484-thread-1-processing-n:127.0.0.1:34073_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 2251187 INFO  (coreZkRegister-10484-thread-1-processing-n:127.0.0.1:34073_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/control_collection/leaders/shard1/leader after winning as /collections/control_collection/leader_elect/shard1/election/96393081490702341-core_node1-n_0000000000
   [junit4]   2> 2251189 INFO  (coreZkRegister-10484-thread-1-processing-n:127.0.0.1:34073_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:34073/collection1/ shard1
   [junit4]   2> 2251190 INFO  (OverseerStateUpdate-96393081490702341-127.0.0.1:34073_-n_0000000000) [n:127.0.0.1:34073_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"https://127.0.0.1:34073",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 1
   [junit4]   2> 2251292 INFO  (zkCallback-32732-thread-2-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 2251339 INFO  (coreZkRegister-10484-thread-1-processing-n:127.0.0.1:34073_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are https://127.0.0.1:34073/collection1/ and leader is https://127.0.0.1:34073/collection1/
   [junit4]   2> 2251340 INFO  (coreZkRegister-10484-thread-1-processing-n:127.0.0.1:34073_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:34073
   [junit4]   2> 2251340 INFO  (coreZkRegister-10484-thread-1-processing-n:127.0.0.1:34073_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 2251340 INFO  (coreZkRegister-10484-thread-1-processing-n:127.0.0.1:34073_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 2251340 INFO  (coreZkRegister-10484-thread-1-processing-n:127.0.0.1:34073_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 2251341 INFO  (coreZkRegister-10484-thread-1-processing-n:127.0.0.1:34073_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Checking legacy cluster state for collection control_collection
   [junit4]   2> 2251341 INFO  (OverseerStateUpdate-96393081490702341-127.0.0.1:34073_-n_0000000000) [n:127.0.0.1:34073_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:34073",
   [junit4]   2>   "node_name":"127.0.0.1:34073_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"} current state version: 2
   [junit4]   2> 2251341 INFO  (OverseerStateUpdate-96393081490702341-127.0.0.1:34073_-n_0000000000) [n:127.0.0.1:34073_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:34073",
   [junit4]   2>   "node_name":"127.0.0.1:34073_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 2251414 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2251419 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2251427 INFO  (zkCallback-32736-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2de06f22 name:ZooKeeperConnection Watcher:127.0.0.1:37694/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2251427 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2251428 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2251428 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 2251428 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.c.ZkStateReader Loaded cluster properties: {urlScheme=https}
   [junit4]   2> 2251428 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 2251429 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 2251454 INFO  (zkCallback-32736-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 2251455 INFO  (zkCallback-32732-thread-2-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 2251542 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A64F024E870E254C-001/shard-1-001/cores/collection1
   [junit4]   2> 2251543 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A64F024E870E254C-001/shard-1-001
   [junit4]   2> 2251545 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.e.j.s.Server jetty-9.3.8.v20160314
   [junit4]   2> 2251610 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@65d23177{/,null,AVAILABLE}
   [junit4]   2> 2251611 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.e.j.s.ServerConnector Started ServerConnector@6e4056a6{SSL,[ssl, http/1.1]}{127.0.0.1:42054}
   [junit4]   2> 2251611 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.e.j.s.Server Started @2254298ms
   [junit4]   2> 2251611 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A64F024E870E254C-001/tempDir-001/jetty1, solrconfig=solrconfig.xml, hostContext=/, hostPort=42054, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A64F024E870E254C-001/shard-1-001/cores}
   [junit4]   2> 2251612 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@73d16e93
   [junit4]   2> 2251612 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A64F024E870E254C-001/shard-1-001'
   [junit4]   2> 2251612 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 2251612 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 2251612 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2251627 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2251634 INFO  (zkCallback-32737-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5ef792bc name:ZooKeeperConnection Watcher:127.0.0.1:37694/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2251634 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2251635 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2251636 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 2251636 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A64F024E870E254C-001/shard-1-001/solr.xml
   [junit4]   2> 2251641 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A64F024E870E254C-001/shard-1-001/cores
   [junit4]   2> 2251641 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.CoreContainer New CoreContainer 495292304
   [junit4]   2> 2251641 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A64F024E870E254C-001/shard-1-001]
   [junit4]   2> 2251641 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A64F024E870E254C-001/shard-1-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A64F024E870E254C-001/shard-1-001/lib
   [junit4]   2> 2251641 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,connectionsEvictorSleepDelay : 5000,maxConnectionIdleTime : 40000,
   [junit4]   2> 2251649 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 2251657 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 2251658 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 2251658 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:37694/solr
   [junit4]   2> 2251658 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 2251658 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2251666 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2251677 INFO  (zkCallback-32740-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4d82fed9 name:ZooKeeperConnection Watcher:127.0.0.1:37694 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2251678 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2251678 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2251679 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [n:127.0.0.1:42054_    ] o.a.s.c.ZkController Added new OnReconnect listener org.apache.solr.cloud.ZkController$$Lambda$12/2101902147@60ef4c78
   [junit4]   2> 2251697 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [n:127.0.0.1:42054_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2251702 INFO  (zkCallback-32741-thread-1-processing-n:127.0.0.1:42054_) [n:127.0.0.1:42054_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2f7c1d5a name:ZooKeeperConnection Watcher:127.0.0.1:37694/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2251703 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [n:127.0.0.1:42054_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2251706 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [n:127.0.0.1:42054_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 2251707 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [n:127.0.0.1:42054_    ] o.a.s.c.c.ZkStateReader Loaded cluster properties: {urlScheme=https}
   [junit4]   2> 2251707 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [n:127.0.0.1:42054_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 2251709 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [n:127.0.0.1:42054_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 2251709 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [n:127.0.0.1:42054_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/96393081490702345-127.0.0.1:42054_-n_0000000001
   [junit4]   2> 2251709 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [n:127.0.0.1:42054_    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/96393081490702341-127.0.0.1:34073_-n_0000000000 to know if I could be the leader
   [junit4]   2> 2251710 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [n:127.0.0.1:42054_    ] o.a.s.c.ZkController Publish node=127.0.0.1:42054_ as DOWN
   [junit4]   2> 2251710 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [n:127.0.0.1:42054_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:42054_
   [junit4]   2> 2251710 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [n:127.0.0.1:42054_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:42054_
   [junit4]   2> 2251711 INFO  (OverseerStateUpdate-96393081490702341-127.0.0.1:34073_-n_0000000000) [n:127.0.0.1:34073_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"downnode",
   [junit4]   2>   "node_name":"127.0.0.1:42054_"} current state version: 3
   [junit4]   2> 2251711 INFO  (OverseerStateUpdate-96393081490702341-127.0.0.1:34073_-n_0000000000) [n:127.0.0.1:34073_    ] o.a.s.c.o.NodeMutator DownNode state invoked for node: 127.0.0.1:42054_
   [junit4]   2> 2251711 INFO  (zkCallback-32736-thread-1) [    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 2251711 INFO  (zkCallback-32732-thread-2-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 2251711 INFO  (zkCallback-32741-thread-1-processing-n:127.0.0.1:42054_) [n:127.0.0.1:42054_    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 2251712 INFO  (zkCallback-32736-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 2251712 INFO  (zkCallback-32732-thread-2-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 2251715 INFO  (zkCallback-32741-thread-1-processing-n:127.0.0.1:42054_) [n:127.0.0.1:42054_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 2251751 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [n:127.0.0.1:42054_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 2251751 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [n:127.0.0.1:42054_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 2251752 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [n:127.0.0.1:42054_    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A64F024E870E254C-001/shard-1-001/cores
   [junit4]   2> 2251753 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [n:127.0.0.1:42054_    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, loadOnStartup=true, configSetProperties=configsetprops.json, collection=collection1, coreNodeName=, dataDir=data/, shard=}
   [junit4]   2> 2251753 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [n:127.0.0.1:42054_    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A64F024E870E254C-001/shard-1-001/cores/collection1
   [junit4]   2> 2251753 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [n:127.0.0.1:42054_    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 2251766 INFO  (coreLoadExecutor-10501-thread-1-processing-n:127.0.0.1:42054_) [n:127.0.0.1:42054_ c:collection1   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 2251766 INFO  (coreLoadExecutor-10501-thread-1-processing-n:127.0.0.1:42054_) [n:127.0.0.1:42054_ c:collection1   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 2251767 INFO  (coreLoadExecutor-10501-thread-1-processing-n:127.0.0.1:42054_) [n:127.0.0.1:42054_ c:collection1   x:collection1] o.a.s.c.ZkController Collection collection1 not visible yet, but flagging it so a watch is registered when it becomes visible
   [junit4]   2> 2251767 INFO  (OverseerStateUpdate-96393081490702341-127.0.0.1:34073_-n_0000000000) [n:127.0.0.1:34073_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:42054",
   [junit4]   2>   "node_name":"127.0.0.1:42054_",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "operation":"state"} current state version: 3
   [junit4]   2> 2251767 INFO  (OverseerStateUpdate-96393081490702341-127.0.0.1:34073_-n_0000000000) [n:127.0.0.1:34073_    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:42054",
   [junit4]   2>   "node_name":"127.0.0.1:42054_",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 2251767 INFO  (OverseerStateUpdate-96393081490702341-127.0.0.1:34073_-n_0000000000) [n:127.0.0.1:34073_    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 2251767 INFO  (OverseerStateUpdate-96393081490702341-127.0.0.1:34073_-n_0000000000) [n:127.0.0.1:34073_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2
   [junit4]   2> 2251767 INFO  (coreLoadExecutor-10501-thread-1-processing-n:127.0.0.1:42054_) [n:127.0.0.1:42054_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Deleting data for [collection1]
   [junit4]   2> 2251788 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [n:127.0.0.1:42054_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1
   [junit4]   2> 2251788 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [n:127.0.0.1:42054_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 2251798 INFO  (coreLoadExecutor-10501-thread-1-processing-n:127.0.0.1:42054_) [n:127.0.0.1:42054_ c:collection1   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 2251869 INFO  (zkCallback-32736-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2251869 INFO  (zkCallback-32732-thread-2-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2251869 INFO  (zkCallback-32741-thread-1-processing-n:127.0.0.1:42054_) [n:127.0.0.1:42054_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2252798 INFO  (coreLoadExecutor-10501-thread-1-processing-n:127.0.0.1:42054_) [n:127.0.0.1:42054_ c:collection1   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 2252798 INFO  (coreLoadExecutor-10501-thread-1-processing-n:127.0.0.1:42054_) [n:127.0.0.1:42054_ c:collection1   x:collection1] o.a.s.c.ZkController Check for collection zkNode:collection1
   [junit4]   2> 2252799 INFO  (coreLoadExecutor-10501-thread-1-processing-n:127.0.0.1:42054_) [n:127.0.0.1:42054_ c:collection1   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 2252799 INFO  (coreLoadExecutor-10501-thread-1-processing-n:127.0.0.1:42054_) [n:127.0.0.1:42054_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/collection1]
   [junit4]   2> 2252799 INFO  (coreLoadExecutor-10501-thread-1-processing-n:127.0.0.1:42054_) [n:127.0.0.1:42054_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/collection1] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 2252799 INFO  (coreLoadExecutor-10501-thread-1-processing-n:127.0.0.1:42054_) [n:127.0.0.1:42054_ c:collection1   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A64F024E870E254C-001/shard-1-001/cores/collection1'
   [junit4]   2> 2252799 INFO  (coreLoadExecutor-10501-thread-1-processing-n:127.0.0.1:42054_) [n:127.0.0.1:42054_ c:collection1   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 2252799 INFO  (coreLoadExecutor-10501-thread-1-processing-n:127.0.0.1:42054_) [n:127.0.0.1:42054_ c:collection1   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 2252801 INFO  (coreLoadExecutor-10501-thread-1-processing-n:127.0.0.1:42054_) [n:127.0.0.1:42054_ c:collection1   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 2252805 INFO  (coreLoadExecutor-10501-thread-1-processing-n:127.0.0.1:42054_) [n:127.0.0.1:42054_ c:collection1   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 2252808 WARN  (coreLoadExecutor-10501-thread-1-processing-n:127.0.0.1:42054_) [n:127.0.0.1:42054_ c:collection1   x:collection1] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
   [junit4]   2> 2252809 INFO  (coreLoadExecutor-10501-thread-1-processing-n:127.0.0.1:42054_) [n:127.0.0.1:42054_ c:collection1   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.2.0
   [junit4]   2> 2252833 INFO  (coreLoadExecutor-10501-thread-1-processing-n:127.0.0.1:42054_) [n:127.0.0.1:42054_ c:collection1   x:collection1] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 2252839 INFO  (coreLoadExecutor-10501-thread-1-processing-n:127.0.0.1:42054_) [n:127.0.0.1:42054_ c:collection1   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 2253033 INFO  (coreLoadExecutor-10501-thread-1-processing-n:127.0.0.1:42054_) [n:127.0.0.1:42054_ c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 2253039 INFO  (coreLoadExecutor-10501-thread-1-processing-n:127.0.0.1:42054_) [n:127.0.0.1:42054_ c:collection1   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 2253040 INFO  (coreLoadExecutor-10501-thread-1-processing-n:127.0.0.1:42054_) [n:127.0.0.1:42054_ c:collection1   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 2253049 INFO  (coreLoadExecutor-10501-thread-1-processing-n:127.0.0.1:42054_) [n:127.0.0.1:42054_ c:collection1   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 2253050 INFO  (coreLoadExecutor-10501-thread-1-processing-n:127.0.0.1:42054_) [n:127.0.0.1:42054_ c:collection1   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 2253062 INFO  (coreLoadExecutor-10501-thread-1-processing-n:127.0.0.1:42054_) [n:127.0.0.1:42054_ c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 2253062 INFO  (coreLoadExecutor-10501-thread-1-processing-n:127.0.0.1:42054_) [n:127.0.0.1:42054_ c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 2253064 INFO  (coreLoadExecutor-10501-thread-1-processing-n:127.0.0.1:42054_) [n:127.0.0.1:42054_ c:collection1   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1
   [junit4]   2> 2253064 INFO  (coreLoadExecutor-10501-thread-1-processing-n:127.0.0.1:42054_) [n:127.0.0.1:42054_ c:collection1   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' 

[...truncated too long message...]

s-6.x/solr/core/test-lib/apacheds-interceptors-schema-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-subtree-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-trigger-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-jdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-jdbm1-2.0.0-M2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-kerberos-codec-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-ldif-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-mavibot-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-protocol-kerberos-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-protocol-ldap-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-protocol-shared-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-xdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/api-all-1.0.0-M20.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/bcprov-jdk15-1.45.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/cglib-nodep-2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/commons-collections-3.2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/commons-math3-3.4.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/easymock-3.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/ehcache-core-2.4.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/hadoop-common-2.7.2-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/hadoop-hdfs-2.7.2-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/hadoop-minikdc-2.7.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/htrace-core-3.2.0-incubating.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jackson-annotations-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jackson-core-asl-1.9.13.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jackson-databind-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jackson-mapper-asl-1.9.13.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jersey-core-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jersey-server-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jetty-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jetty-util-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/mina-core-2.0.0-M5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/mockito-core-1.9.5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/netty-all-4.0.36.Final.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/objenesis-1.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/presto-parser-0.122.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/slice-0.10.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/analysis/icu/lucene-analyzers-icu-6.2.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/contrib/solr-analysis-extras/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/queryparser/classes/test:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/contrib/analysis-extras/lib/icu4j-56.1.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-launcher.jar:/x1/jenkins/.ant/lib/ivy-2.3.0.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-antlr.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-regexp.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-resolver.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-net.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jmf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-oro.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-xalan2.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-logging.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jsch.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit4.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-javamail.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jai.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jdepend.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bsf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-swing.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-netrexx.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bcel.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-testutil.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-log4j.jar:/x1/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.8/lib/tools.jar:/x1/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.3.4.jar com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/temp/junit4-J2-20160810_143044_914.events @/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/temp/junit4-J2-20160810_143044_914.suites -stdin
   [junit4] ERROR: JVM J2 ended with an exception: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1501)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:121)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:955)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:952)
   [junit4] 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4] 	at java.lang.Thread.run(Thread.java:745)

BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:771: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:707: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/common-build.xml:536: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/common-build.xml:1443: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/common-build.xml:984: At least one slave process threw an exception, first: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.

Total time: 265 minutes 49 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
ERROR: Failed to archive {README.txt=README.txt, solr/build/solr-core/test/temp/junit4-J2-20160810_143044_914.events=solr/build/solr-core/test/temp/junit4-J2-20160810_143044_914.events, heapdumps/java_pid25425.hprof=heapdumps/java_pid25425.hprof, solr/build/solr-core/test/temp/junit4-J0-20160810_143044_911.events=solr/build/solr-core/test/temp/junit4-J0-20160810_143044_911.events, solr/build/solr-core/test/temp/junit4-J1-20160810_143044_911.events=solr/build/solr-core/test/temp/junit4-J1-20160810_143044_911.events} due to internal error; falling back to full archiving
java.lang.NullPointerException
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any




[JENKINS] Lucene-Solr-NightlyTests-6.x - Build # 141 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-6.x/141/

3 tests failed.
FAILED:  org.apache.solr.cloud.ForceLeaderTest.testReplicasInLIRNoLeader

Error Message:
No live SolrServers available to handle this request:[http://127.0.0.1:46010/forceleader_test_collection_shard1_replica1]

Stack Trace:
org.apache.solr.client.solrj.impl.CloudSolrClient$RouteException: No live SolrServers available to handle this request:[http://127.0.0.1:46010/forceleader_test_collection_shard1_replica1]
	at __randomizedtesting.SeedInfo.seed([6F667FE33CB5C521:89F14B2305373C40]:0)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.directUpdate(CloudSolrClient.java:760)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1172)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1061)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:997)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.sendDocsWithRetry(AbstractFullDistribZkTestBase.java:753)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.sendDocsWithRetry(AbstractFullDistribZkTestBase.java:741)
	at org.apache.solr.cloud.ForceLeaderTest.sendDoc(ForceLeaderTest.java:424)
	at org.apache.solr.cloud.ForceLeaderTest.testReplicasInLIRNoLeader(ForceLeaderTest.java:131)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:46010/forceleader_test_collection_shard1_replica1]
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:383)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.lambda$directUpdate$0(CloudSolrClient.java:735)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	... 1 more
Caused by: org.apache.solr.client.solrj.SolrServerException: Server refused connection at: http://127.0.0.1:46010/forceleader_test_collection_shard1_replica1
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:599)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:261)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:250)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:403)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:372)
	... 6 more
Caused by: java.net.ConnectException: Connection refused
	at java.net.PlainSocketImpl.socketConnect(Native Method)
	at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
	at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
	at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
	at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
	at java.net.Socket.connect(Socket.java:589)
	at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:117)
	at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:177)
	at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:304)
	at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:611)
	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:446)
	at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:497)
	... 10 more


FAILED:  org.apache.solr.cloud.hdfs.HdfsSyncSliceTest.test

Error Message:
There are still nodes recoverying - waited for 30 seconds

Stack Trace:
java.lang.AssertionError: There are still nodes recoverying - waited for 30 seconds
	at __randomizedtesting.SeedInfo.seed([6F667FE33CB5C521:E73240399249A8D9]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:182)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.waitForRecoveriesToFinish(AbstractFullDistribZkTestBase.java:862)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.waitForThingsToLevelOut(AbstractFullDistribZkTestBase.java:1418)
	at org.apache.solr.cloud.SyncSliceTest.test(SyncSliceTest.java:71)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.cloud.CdcrReplicationDistributedZkTest.testReplicationAfterRestart

Error Message:
Timeout waiting for CDCR replication to complete @source_collection:shard1

Stack Trace:
java.lang.RuntimeException: Timeout waiting for CDCR replication to complete @source_collection:shard1
	at __randomizedtesting.SeedInfo.seed([6F667FE33CB5C521:337B9862F2957117]:0)
	at org.apache.solr.cloud.BaseCdcrDistributedZkTest.waitForReplicationToComplete(BaseCdcrDistributedZkTest.java:794)
	at org.apache.solr.cloud.CdcrReplicationDistributedZkTest.testReplicationAfterRestart(CdcrReplicationDistributedZkTest.java:264)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 11474 lines...]
   [junit4] Suite: org.apache.solr.cloud.hdfs.HdfsSyncSliceTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsSyncSliceTest_6F667FE33CB5C521-001/init-core-data-001
   [junit4]   2> 793576 INFO  (SUITE-HdfsSyncSliceTest-seed#[6F667FE33CB5C521]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
   [junit4]   2> 793578 INFO  (SUITE-HdfsSyncSliceTest-seed#[6F667FE33CB5C521]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /msn/my
   [junit4]   1> Formatting using clusterid: testClusterID
   [junit4]   2> 793610 WARN  (SUITE-HdfsSyncSliceTest-seed#[6F667FE33CB5C521]-worker) [    ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
   [junit4]   2> 793616 WARN  (SUITE-HdfsSyncSliceTest-seed#[6F667FE33CB5C521]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 793618 INFO  (SUITE-HdfsSyncSliceTest-seed#[6F667FE33CB5C521]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 793625 INFO  (SUITE-HdfsSyncSliceTest-seed#[6F667FE33CB5C521]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/hdfs to ./temp/Jetty_localhost_61000_hdfs____.y1o7nl/webapp
   [junit4]   2> 793938 INFO  (SUITE-HdfsSyncSliceTest-seed#[6F667FE33CB5C521]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:61000
   [junit4]   2> 794006 WARN  (SUITE-HdfsSyncSliceTest-seed#[6F667FE33CB5C521]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 794007 INFO  (SUITE-HdfsSyncSliceTest-seed#[6F667FE33CB5C521]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 794018 INFO  (SUITE-HdfsSyncSliceTest-seed#[6F667FE33CB5C521]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_58608_datanode____7pu46a/webapp
   [junit4]   2> 794422 INFO  (SUITE-HdfsSyncSliceTest-seed#[6F667FE33CB5C521]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:58608
   [junit4]   2> 794459 WARN  (SUITE-HdfsSyncSliceTest-seed#[6F667FE33CB5C521]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 794459 INFO  (SUITE-HdfsSyncSliceTest-seed#[6F667FE33CB5C521]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 794471 INFO  (SUITE-HdfsSyncSliceTest-seed#[6F667FE33CB5C521]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_40843_datanode____mj7y1u/webapp
   [junit4]   2> 794546 INFO  (IPC Server handler 3 on 43159) [    ] BlockStateChange BLOCK* processReport: from storage DS-82809e76-64b9-4878-8d10-18a57e1357ef node DatanodeRegistration(127.0.0.1:34154, datanodeUuid=052daa0d-a8e9-4386-b794-88085936c789, infoPort=37636, infoSecurePort=0, ipcPort=41893, storageInfo=lv=-56;cid=testClusterID;nsid=447807271;c=0), blocks: 0, hasStaleStorage: true, processing time: 0 msecs
   [junit4]   2> 794546 INFO  (IPC Server handler 3 on 43159) [    ] BlockStateChange BLOCK* processReport: from storage DS-c61bfa81-06e9-4aad-9d0c-583bece9b712 node DatanodeRegistration(127.0.0.1:34154, datanodeUuid=052daa0d-a8e9-4386-b794-88085936c789, infoPort=37636, infoSecurePort=0, ipcPort=41893, storageInfo=lv=-56;cid=testClusterID;nsid=447807271;c=0), blocks: 0, hasStaleStorage: false, processing time: 0 msecs
   [junit4]   2> 794839 INFO  (SUITE-HdfsSyncSliceTest-seed#[6F667FE33CB5C521]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:40843
   [junit4]   2> 794908 INFO  (IPC Server handler 8 on 43159) [    ] BlockStateChange BLOCK* processReport: from storage DS-d52833c7-fb03-4b33-acaa-06001e10c330 node DatanodeRegistration(127.0.0.1:37007, datanodeUuid=171cdc8a-684f-47d8-9345-0d5235857205, infoPort=47138, infoSecurePort=0, ipcPort=53015, storageInfo=lv=-56;cid=testClusterID;nsid=447807271;c=0), blocks: 0, hasStaleStorage: true, processing time: 0 msecs
   [junit4]   2> 794909 INFO  (IPC Server handler 8 on 43159) [    ] BlockStateChange BLOCK* processReport: from storage DS-d28cdf0d-481d-4d39-b5a9-4cbd7cced32f node DatanodeRegistration(127.0.0.1:37007, datanodeUuid=171cdc8a-684f-47d8-9345-0d5235857205, infoPort=47138, infoSecurePort=0, ipcPort=53015, storageInfo=lv=-56;cid=testClusterID;nsid=447807271;c=0), blocks: 0, hasStaleStorage: false, processing time: 0 msecs
   [junit4]   2> 794977 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 794977 INFO  (Thread-6000) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 794978 INFO  (Thread-6000) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 795078 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.ZkTestServer start zk server on port:59746
   [junit4]   2> 795078 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 795078 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 795080 INFO  (zkCallback-547-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@70454e0f name:ZooKeeperConnection Watcher:127.0.0.1:59746 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 795080 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 795080 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 795080 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 795082 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 795082 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 795083 INFO  (zkCallback-548-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@61baffb4 name:ZooKeeperConnection Watcher:127.0.0.1:59746/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 795083 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 795083 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 795083 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 795084 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 795085 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 795085 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 795086 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 795086 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 795087 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 795087 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 795088 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 795088 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 795089 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 795089 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 795090 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 795090 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 795091 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 795091 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 795091 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 795092 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 795092 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 795092 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 795093 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 795093 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 795094 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 795094 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 795095 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 795095 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 795096 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 795096 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 795096 INFO  (zkCallback-550-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@693a793e name:ZooKeeperConnection Watcher:127.0.0.1:59746/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 795097 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 795097 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 795157 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsSyncSliceTest_6F667FE33CB5C521-001/control-001/cores/collection1
   [junit4]   2> 795158 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.e.j.s.Server jetty-9.3.8.v20160314
   [junit4]   2> 795160 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@574f60da{/msn/my,null,AVAILABLE}
   [junit4]   2> 795161 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.e.j.s.ServerConnector Started ServerConnector@14b103c9{SSL,[ssl, http/1.1]}{127.0.0.1:57949}
   [junit4]   2> 795161 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.e.j.s.Server Started @797539ms
   [junit4]   2> 795162 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=hdfs://localhost:43159/hdfs__localhost_43159__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-6.x_solr_build_solr-core_test_J1_temp_solr.cloud.hdfs.HdfsSyncSliceTest_6F667FE33CB5C521-001_tempDir-002_control_data, hostContext=/msn/my, hostPort=57949, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsSyncSliceTest_6F667FE33CB5C521-001/control-001/cores}
   [junit4]   2> 795162 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@73d16e93
   [junit4]   2> 795162 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsSyncSliceTest_6F667FE33CB5C521-001/control-001'
   [junit4]   2> 795162 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 795162 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 795162 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 795162 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 795163 INFO  (zkCallback-551-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4c07871f name:ZooKeeperConnection Watcher:127.0.0.1:59746/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 795163 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 795163 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 795164 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 795164 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsSyncSliceTest_6F667FE33CB5C521-001/control-001/solr.xml
   [junit4]   2> 795167 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsSyncSliceTest_6F667FE33CB5C521-001/control-001/cores
   [junit4]   2> 795167 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.CoreContainer New CoreContainer 1356455593
   [junit4]   2> 795168 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsSyncSliceTest_6F667FE33CB5C521-001/control-001]
   [junit4]   2> 795168 WARN  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsSyncSliceTest_6F667FE33CB5C521-001/control-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsSyncSliceTest_6F667FE33CB5C521-001/control-001/lib
   [junit4]   2> 795168 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,connectionsEvictorSleepDelay : 5000,maxConnectionIdleTime : 40000,
   [junit4]   2> 795169 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 795169 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 795169 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 795169 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:59746/solr
   [junit4]   2> 795169 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 795169 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 795169 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 795170 INFO  (zkCallback-554-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@f403764 name:ZooKeeperConnection Watcher:127.0.0.1:59746 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 795170 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 795171 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 795171 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [n:127.0.0.1:57949_msn%2Fmy    ] o.a.s.c.ZkController Added new OnReconnect listener org.apache.solr.cloud.ZkController$$Lambda$79/1447441381@1b911e2c
   [junit4]   2> 795171 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [n:127.0.0.1:57949_msn%2Fmy    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 795172 INFO  (zkCallback-555-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2afb88a1 name:ZooKeeperConnection Watcher:127.0.0.1:59746/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 795172 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [n:127.0.0.1:57949_msn%2Fmy    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 795172 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [n:127.0.0.1:57949_msn%2Fmy    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 795173 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [n:127.0.0.1:57949_msn%2Fmy    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 795175 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [n:127.0.0.1:57949_msn%2Fmy    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 795175 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [n:127.0.0.1:57949_msn%2Fmy    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 795176 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [n:127.0.0.1:57949_msn%2Fmy    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 795177 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [n:127.0.0.1:57949_msn%2Fmy    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 795178 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [n:127.0.0.1:57949_msn%2Fmy    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 795178 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [n:127.0.0.1:57949_msn%2Fmy    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 795179 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [n:127.0.0.1:57949_msn%2Fmy    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 795179 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [n:127.0.0.1:57949_msn%2Fmy    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 795179 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [n:127.0.0.1:57949_msn%2Fmy    ] o.a.s.c.c.ZkStateReader Loaded cluster properties: {urlScheme=https}
   [junit4]   2> 795180 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [n:127.0.0.1:57949_msn%2Fmy    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (0)
   [junit4]   2> 795181 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [n:127.0.0.1:57949_msn%2Fmy    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 795181 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [n:127.0.0.1:57949_msn%2Fmy    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 795182 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [n:127.0.0.1:57949_msn%2Fmy    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 795182 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [n:127.0.0.1:57949_msn%2Fmy    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/96388080370057221-127.0.0.1:57949_msn%2Fmy-n_0000000000
   [junit4]   2> 795182 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [n:127.0.0.1:57949_msn%2Fmy    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:57949_msn%2Fmy
   [junit4]   2> 795183 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [n:127.0.0.1:57949_msn%2Fmy    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 795183 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [n:127.0.0.1:57949_msn%2Fmy    ] o.a.s.c.Overseer Overseer (id=96388080370057221-127.0.0.1:57949_msn%2Fmy-n_0000000000) starting
   [junit4]   2> 795184 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [n:127.0.0.1:57949_msn%2Fmy    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 795191 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [n:127.0.0.1:57949_msn%2Fmy    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 795192 INFO  (OverseerCollectionConfigSetProcessor-96388080370057221-127.0.0.1:57949_msn%2Fmy-n_0000000000) [n:127.0.0.1:57949_msn%2Fmy    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 795192 INFO  (OverseerStateUpdate-96388080370057221-127.0.0.1:57949_msn%2Fmy-n_0000000000) [n:127.0.0.1:57949_msn%2Fmy    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 795192 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [n:127.0.0.1:57949_msn%2Fmy    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:57949_msn%2Fmy
   [junit4]   2> 795192 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [n:127.0.0.1:57949_msn%2Fmy    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:57949_msn%2Fmy
   [junit4]   2> 795194 INFO  (zkCallback-555-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [0])
   [junit4]   2> 795194 INFO  (OverseerStateUpdate-96388080370057221-127.0.0.1:57949_msn%2Fmy-n_0000000000) [n:127.0.0.1:57949_msn%2Fmy    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 795195 INFO  (zkCallback-555-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (1)
   [junit4]   2> 795202 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [n:127.0.0.1:57949_msn%2Fmy    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 795202 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [n:127.0.0.1:57949_msn%2Fmy    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 795202 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [n:127.0.0.1:57949_msn%2Fmy    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsSyncSliceTest_6F667FE33CB5C521-001/control-001/cores
   [junit4]   2> 795203 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [n:127.0.0.1:57949_msn%2Fmy    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, loadOnStartup=true, configSetProperties=configsetprops.json, collection=control_collection, coreNodeName=, dataDir=data/, shard=}
   [junit4]   2> 795203 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [n:127.0.0.1:57949_msn%2Fmy    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsSyncSliceTest_6F667FE33CB5C521-001/control-001/cores/collection1
   [junit4]   2> 795203 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [n:127.0.0.1:57949_msn%2Fmy    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 795203 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 795203 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 795203 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [n:127.0.0.1:57949_msn%2Fmy    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1
   [junit4]   2> 795203 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [n:127.0.0.1:57949_msn%2Fmy    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 795204 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection   x:collection1] o.a.s.c.ZkController Collection control_collection not visible yet, but flagging it so a watch is registered when it becomes visible
   [junit4]   2> 795204 INFO  (OverseerStateUpdate-96388080370057221-127.0.0.1:57949_msn%2Fmy-n_0000000000) [n:127.0.0.1:57949_msn%2Fmy    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:57949/msn/my",
   [junit4]   2>   "node_name":"127.0.0.1:57949_msn%2Fmy",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 795204 INFO  (OverseerStateUpdate-96388080370057221-127.0.0.1:57949_msn%2Fmy-n_0000000000) [n:127.0.0.1:57949_msn%2Fmy    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:57949/msn/my",
   [junit4]   2>   "node_name":"127.0.0.1:57949_msn%2Fmy",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 795204 INFO  (OverseerStateUpdate-96388080370057221-127.0.0.1:57949_msn%2Fmy-n_0000000000) [n:127.0.0.1:57949_msn%2Fmy    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 795204 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Deleting data for [control_collection]
   [junit4]   2> 795204 INFO  (OverseerStateUpdate-96388080370057221-127.0.0.1:57949_msn%2Fmy-n_0000000000) [n:127.0.0.1:57949_msn%2Fmy    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 795205 INFO  (zkCallback-555-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 795205 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 796206 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 796206 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 796206 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 796206 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/control_collection]
   [junit4]   2> 796207 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/control_collection] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 796207 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsSyncSliceTest_6F667FE33CB5C521-001/control-001/cores/collection1'
   [junit4]   2> 796207 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 796207 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 796209 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 796212 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 796215 WARN  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection   x:collection1] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
   [junit4]   2> 796216 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.2.0
   [junit4]   2> 796223 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection   x:collection1] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 796227 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 796304 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 796308 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 796309 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 796313 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 796314 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 796315 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 796316 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 796316 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1
   [junit4]   2> 796316 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 796317 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.HdfsDirectoryFactory
   [junit4]   2> 796317 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:43159/solr_hdfs_home
   [junit4]   2> 796317 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
   [junit4]   2> 796317 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsSyncSliceTest_6F667FE33CB5C521-001/control-001/cores/collection1], dataDir=[hdfs://localhost:43159/solr_hdfs_home/control_collection/core_node1/data/]
   [junit4]   2> 796317 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@732ae4db
   [junit4]   2> 796317 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:43159/solr_hdfs_home/control_collection/core_node1/data/snapshot_metadata
   [junit4]   2> 796322 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
   [junit4]   2> 796322 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
   [junit4]   2> 796322 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
   [junit4]   2> 796329 WARN  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.h.HdfsDirectory The NameNode is in SafeMode - Solr will wait 5 seconds and try again.
   [junit4]   2> 801330 WARN  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.h.HdfsDirectory The NameNode is in SafeMode - Solr will wait 5 seconds and try again.
   [junit4]   2> 806333 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
   [junit4]   2> 806334 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:43159/solr_hdfs_home/control_collection/core_node1/data/snapshot_metadata
   [junit4]   2> 806334 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.s.SolrSnapshotMetaDataManager Loading from snapshot metadata file...
   [junit4]   2> 806334 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:43159/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 806342 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:43159/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 806342 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=hdfs://localhost:43159/solr_hdfs_home/control_collection/core_node1/data/index/
   [junit4]   2> 806348 WARN  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory 'hdfs:/localhost:43159/solr_hdfs_home/control_collection/core_node1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 806348 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:43159/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 806354 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
   [junit4]   2> 806354 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
   [junit4]   2> 806354 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
   [junit4]   2> 806361 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
   [junit4]   2> 806361 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:43159/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 806361 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@6e4f3aeb
   [junit4]   2> 806372 INFO  (IPC Server handler 2 on 43159) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:37007 is added to blk_1073741825_1001{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-c61bfa81-06e9-4aad-9d0c-583bece9b712:NORMAL:127.0.0.1:34154|RBW], ReplicaUC[[DISK]DS-d52833c7-fb03-4b33-acaa-06001e10c330:NORMAL:127.0.0.1:37007|FINALIZED]]} size 0
   [junit4]   2> 806373 INFO  (IPC Server handler 1 on 43159) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:34154 is added to blk_1073741825_1001{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-d52833c7-fb03-4b33-acaa-06001e10c330:NORMAL:127.0.0.1:37007|FINALIZED], ReplicaUC[[DISK]DS-82809e76-64b9-4878-8d10-18a57e1357ef:NORMAL:127.0.0.1:34154|FINALIZED]]} size 0
   [junit4]   2> 806374 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(BlockDirectory(HdfsDirectory@hdfs://localhost:43159/solr_hdfs_home/control_collection/core_node1/data/index lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@944aa68); maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 806374 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 806375 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in hdfs://localhost:43159/solr_hdfs_home/control_collection/core_node1/data/
   [junit4]   2> 806377 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 806377 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 806377 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 806378 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 806378 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 806378 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 806378 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 806378 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 806378 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 806378 WARN  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
   [junit4]   2> 806379 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 806380 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 806380 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 806381 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 806383 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/mbeans,standard,/update/json/docs,/admin/luke,/export,/get,/admin/properties,/update/json,/admin/threads,/dump,/update/csv,/sql,/graph,/admin/segments,/admin/system,/replication,/config,/stream,/schema,/admin/plugins,/admin/logging,/admin/ping,/update,/admin/file,/terms
   [junit4]   2> 806383 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 806384 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
   [junit4]   2> 806384 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 806384 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=2
   [junit4]   2> 806400 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 806400 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 806401 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=951729629, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
   [junit4]   2> 806404 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(BlockDirectory(HdfsDirectory@hdfs://localhost:43159/solr_hdfs_home/control_collection/core_node1/data/index lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@944aa68); maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 806404 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 806404 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@96f88d0[collection1] main]
   [junit4]   2> 806404 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/control_collection]
   [junit4]   2> 806405 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/control_collection] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 806405 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 806405 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 806405 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 806405 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 806405 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 806405 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 806405 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 806405 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 806406 INFO  (searcherExecutor-1562-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@96f88d0[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 806406 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 806406 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 806406 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 806406 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1542209297904041984
   [junit4]   2> 806408 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 1.0ms to seed version buckets with highest version 1542209297904041984
   [junit4]   2> 806408 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 806408 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 806409 INFO  (coreZkRegister-1555-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:https://127.0.0.1:57949/msn/my collection:control_collection shard:shard1
   [junit4]   2> 806409 INFO  (coreZkRegister-1555-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 806411 INFO  (coreZkRegister-1555-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/96388080370057221-core_node1-n_0000000000
   [junit4]   2> 806412 INFO  (coreZkRegister-1555-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard=shard1 and weAreReplacement=false and leaderVoteWait=10000
   [junit4]   2> 806413 INFO  (coreZkRegister-1555-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 806413 INFO  (coreZkRegister-1555-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 806413 INFO  (OverseerStateUpdate-96388080370057221-127.0.0.1:57949_msn%2Fmy-n_0000000000) [n:127.0.0.1:57949_msn%2Fmy    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 806413 INFO  (coreZkRegister-1555-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:57949/msn/my/collection1/
   [junit4]   2> 806413 INFO  (coreZkRegister-1555-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 806413 INFO  (coreZkRegister-1555-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy https://127.0.0.1:57949/msn/my/collection1/ has no replicas
   [junit4]   2> 806414 INFO  (zkCallback-555-thread-2-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 806414 INFO  (coreZkRegister-1555-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 806415 INFO  (coreZkRegister-1555-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/control_collection/leaders/shard1/leader after winning as /collections/control_collection/leader_elect/shard1/election/96388080370057221-core_node1-n_0000000000
   [junit4]   2> 806416 INFO  (coreZkRegister-1555-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:57949/msn/my/collection1/ shard1
   [junit4]   2> 806417 INFO  (OverseerStateUpdate-96388080370057221-127.0.0.1:57949_msn%2Fmy-n_0000000000) [n:127.0.0.1:57949_msn%2Fmy    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"https://127.0.0.1:57949/msn/my",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 2
   [junit4]   2> 806518 INFO  (zkCallback-555-thread-2-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 806567 INFO  (coreZkRegister-1555-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are https://127.0.0.1:57949/msn/my/collection1/ and leader is https://127.0.0.1:57949/msn/my/collection1/
   [junit4]   2> 806567 INFO  (coreZkRegister-1555-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:57949/msn/my
   [junit4]   2> 806567 INFO  (coreZkRegister-1555-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 806567 INFO  (coreZkRegister-1555-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 806568 INFO  (coreZkRegister-1555-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 806568 INFO  (coreZkRegister-1555-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Checking legacy cluster state for collection control_collection
   [junit4]   2> 806569 INFO  (OverseerStateUpdate-96388080370057221-127.0.0.1:57949_msn%2Fmy-n_0000000000) [n:127.0.0.1:57949_msn%2Fmy    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "dataDir":"hdfs://localhost:43159/solr_hdfs_home/control_collection/core_node1/data/",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:57949/msn/my",
   [junit4]   2>   "node_name":"127.0.0.1:57949_msn%2Fmy",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "ulogDir":"hdfs://localhost:43159/solr_hdfs_home/control_collection/core_node1/data/tlog"} current state version: 3
   [junit4]   2> 806569 INFO  (OverseerStateUpdate-96388080370057221-127.0.0.1:57949_msn%2Fmy-n_0000000000) [n:127.0.0.1:57949_msn%2Fmy    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "dataDir":"hdfs://localhost:43159/solr_hdfs_home/control_collection/core_node1/data/",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:57949/msn/my",
   [junit4]   2>   "node_name":"127.0.0.1:57949_msn%2Fmy",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "ulogDir":"hdfs://localhost:43159/solr_hdfs_home/control_collection/core_node1/data/tlog"}
   [junit4]   2> 806670 INFO  (zkCallback-555-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 806707 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 806708 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 806709 INFO  (zkCallback-559-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7f9fd71f name:ZooKeeperConnection Watcher:127.0.0.1:59746/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 806709 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 806709 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 806709 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 806709 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.c.ZkStateReader Loaded cluster properties: {urlScheme=https}
   [junit4]   2> 806709 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 806710 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause

[...truncated too long message...]

exception-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-journal-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-normalization-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-operational-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-referral-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-schema-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-subtree-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-trigger-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-jdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-jdbm1-2.0.0-M2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-kerberos-codec-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-ldif-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-mavibot-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-protocol-kerberos-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-protocol-ldap-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-protocol-shared-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-xdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/api-all-1.0.0-M20.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/bcprov-jdk15-1.45.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/cglib-nodep-2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/commons-collections-3.2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/commons-math3-3.4.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/easymock-3.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/ehcache-core-2.4.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/hadoop-common-2.7.2-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/hadoop-hdfs-2.7.2-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/hadoop-minikdc-2.7.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/htrace-core-3.2.0-incubating.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jackson-annotations-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jackson-core-asl-1.9.13.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jackson-databind-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jackson-mapper-asl-1.9.13.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jersey-core-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jersey-server-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jetty-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jetty-util-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/mina-core-2.0.0-M5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/mockito-core-1.9.5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/netty-all-4.0.36.Final.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/objenesis-1.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/presto-parser-0.122.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/slice-0.10.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/analysis/icu/lucene-analyzers-icu-6.2.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/contrib/solr-analysis-extras/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/queryparser/classes/test:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/contrib/analysis-extras/lib/icu4j-56.1.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-launcher.jar:/x1/jenkins/.ant/lib/ivy-2.3.0.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-antlr.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-regexp.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-resolver.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-net.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jmf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-oro.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-xalan2.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-logging.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jsch.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit4.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-javamail.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jai.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jdepend.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bsf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-swing.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-netrexx.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bcel.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-testutil.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-log4j.jar:/x1/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.8/lib/tools.jar:/x1/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.3.4.jar com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/temp/junit4-J0-20160809_174309_109.events @/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/temp/junit4-J0-20160809_174309_109.suites -stdin
   [junit4] ERROR: JVM J0 ended with an exception: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1501)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:121)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:955)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:952)
   [junit4] 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4] 	at java.lang.Thread.run(Thread.java:745)

BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:771: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:707: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/common-build.xml:536: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/common-build.xml:1443: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/common-build.xml:984: At least one slave process threw an exception, first: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.

Total time: 586 minutes 18 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any




[JENKINS] Lucene-Solr-NightlyTests-6.x - Build # 140 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-6.x/140/

2 tests failed.
FAILED:  org.apache.solr.cloud.CdcrReplicationDistributedZkTest.testReplicationAfterRestart

Error Message:
Timeout waiting for CDCR replication to complete @source_collection:shard1

Stack Trace:
java.lang.RuntimeException: Timeout waiting for CDCR replication to complete @source_collection:shard1
	at __randomizedtesting.SeedInfo.seed([C519C4A7A908694E:990423266728DD78]:0)
	at org.apache.solr.cloud.BaseCdcrDistributedZkTest.waitForReplicationToComplete(BaseCdcrDistributedZkTest.java:794)
	at org.apache.solr.cloud.CdcrReplicationDistributedZkTest.testReplicationAfterRestart(CdcrReplicationDistributedZkTest.java:264)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Timeout occured while waiting response from server at: http://127.0.0.1:58224

Stack Trace:
org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:58224
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:603)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:261)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:250)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.makeRequest(CollectionsAPIDistributedZkTest.java:399)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testErrorHandling(CollectionsAPIDistributedZkTest.java:457)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test(CollectionsAPIDistributedZkTest.java:179)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.net.SocketTimeoutException: Read timed out
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
	at java.net.SocketInputStream.read(SocketInputStream.java:170)
	at java.net.SocketInputStream.read(SocketInputStream.java:141)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:160)
	at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:84)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:273)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
	at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
	at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)
	at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)
	at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197)
	at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)
	at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)
	at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685)
	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487)
	at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:497)
	... 47 more




Build Log:
[...truncated 11767 lines...]
   [junit4] JVM J2: stdout was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/temp/junit4-J2-20160808_110251_850.sysout
   [junit4] >>> JVM J2 emitted unexpected output (verbatim) ----
   [junit4] java.lang.OutOfMemoryError: GC overhead limit exceeded
   [junit4] Dumping heap to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/heapdumps/java_pid9110.hprof ...
   [junit4] Heap dump file created [735374596 bytes in 9.294 secs]
   [junit4] <<< JVM J2: EOF ----

   [junit4] JVM J2: stderr was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/temp/junit4-J2-20160808_110251_850.syserr
   [junit4] >>> JVM J2 emitted unexpected output (verbatim) ----
   [junit4] WARN: Unhandled exception in event serialization. -> java.lang.OutOfMemoryError: GC overhead limit exceeded
   [junit4] <<< JVM J2: EOF ----

[...truncated 80 lines...]
   [junit4] Suite: org.apache.solr.cloud.CdcrReplicationDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CdcrReplicationDistributedZkTest_C519C4A7A908694E-001/init-core-data-001
   [junit4]   2> 302829 INFO  (SUITE-CdcrReplicationDistributedZkTest-seed#[C519C4A7A908694E]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
   [junit4]   2> 302830 INFO  (SUITE-CdcrReplicationDistributedZkTest-seed#[C519C4A7A908694E]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   2> 302833 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 302833 INFO  (Thread-789) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 302833 INFO  (Thread-789) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 302933 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.ZkTestServer start zk server on port:41903
   [junit4]   2> 302933 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 302934 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 302937 INFO  (zkCallback-492-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@27e010 name:ZooKeeperConnection Watcher:127.0.0.1:41903 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 302937 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 302937 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 302937 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 302941 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 302941 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 302944 INFO  (zkCallback-493-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5de3642b name:ZooKeeperConnection Watcher:127.0.0.1:41903/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 302944 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 302945 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 302945 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 302947 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 302949 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 302950 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 302952 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-cdcr.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 302952 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 302954 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
   [junit4]   2> 302954 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 302957 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 302957 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 302958 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 302958 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 302960 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 302960 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 302962 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 302962 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 302963 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 302963 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 302966 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 302966 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 302974 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 302974 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 302977 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 302977 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 302979 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 302979 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 302980 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 302981 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 302982 INFO  (zkCallback-495-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2b0f04f3 name:ZooKeeperConnection Watcher:127.0.0.1:41903/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 302982 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 302982 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 302984 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testBufferOnNonLeader
   [junit4]   2> 303053 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CdcrReplicationDistributedZkTest_C519C4A7A908694E-001/jetty-001/cores/collection1
   [junit4]   2> 303055 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.e.j.s.Server jetty-9.3.8.v20160314
   [junit4]   2> 303056 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@62f70b32{/,null,AVAILABLE}
   [junit4]   2> 303064 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.e.j.s.ServerConnector Started ServerConnector@2a76fbe7{SSL,[ssl, http/1.1]}{127.0.0.1:50701}
   [junit4]   2> 303064 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.e.j.s.Server Started @305217ms
   [junit4]   2> 303064 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {shards=shard1, hostContext=/, hostPort=50701, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CdcrReplicationDistributedZkTest_C519C4A7A908694E-001/jetty-001/cores}
   [junit4]   2> 303064 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@73d16e93
   [junit4]   2> 303065 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CdcrReplicationDistributedZkTest_C519C4A7A908694E-001/jetty-001'
   [junit4]   2> 303065 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 303065 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 303065 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 303065 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 303066 INFO  (zkCallback-496-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@57538a16 name:ZooKeeperConnection Watcher:127.0.0.1:41903/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 303067 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 303067 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 303067 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 303068 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CdcrReplicationDistributedZkTest_C519C4A7A908694E-001/jetty-001/solr.xml
   [junit4]   2> 303071 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CdcrReplicationDistributedZkTest_C519C4A7A908694E-001/jetty-001/cores
   [junit4]   2> 303071 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.CoreContainer New CoreContainer 808811889
   [junit4]   2> 303071 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CdcrReplicationDistributedZkTest_C519C4A7A908694E-001/jetty-001]
   [junit4]   2> 303071 WARN  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CdcrReplicationDistributedZkTest_C519C4A7A908694E-001/jetty-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CdcrReplicationDistributedZkTest_C519C4A7A908694E-001/jetty-001/lib
   [junit4]   2> 303071 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,connectionsEvictorSleepDelay : 5000,maxConnectionIdleTime : 40000,
   [junit4]   2> 303072 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 303073 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 303073 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 303073 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:41903/solr
   [junit4]   2> 303073 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 303073 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 303073 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 303077 INFO  (zkCallback-499-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@587827f6 name:ZooKeeperConnection Watcher:127.0.0.1:41903 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 303077 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 303077 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 303078 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [n:127.0.0.1:50701_    ] o.a.s.c.ZkController Added new OnReconnect listener org.apache.solr.cloud.ZkController$$Lambda$10/276052250@950bd52
   [junit4]   2> 303078 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [n:127.0.0.1:50701_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 303079 INFO  (zkCallback-500-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@57aaf1a8 name:ZooKeeperConnection Watcher:127.0.0.1:41903/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 303079 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [n:127.0.0.1:50701_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 303080 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [n:127.0.0.1:50701_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 303082 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [n:127.0.0.1:50701_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 303083 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [n:127.0.0.1:50701_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 303085 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [n:127.0.0.1:50701_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 303086 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [n:127.0.0.1:50701_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 303087 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [n:127.0.0.1:50701_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 303087 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [n:127.0.0.1:50701_    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 303088 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [n:127.0.0.1:50701_    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 303089 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [n:127.0.0.1:50701_    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 303090 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [n:127.0.0.1:50701_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 303090 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [n:127.0.0.1:50701_    ] o.a.s.c.c.ZkStateReader Loaded cluster properties: {urlScheme=https}
   [junit4]   2> 303091 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [n:127.0.0.1:50701_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (0)
   [junit4]   2> 303093 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [n:127.0.0.1:50701_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 303093 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [n:127.0.0.1:50701_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 303095 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [n:127.0.0.1:50701_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 303095 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [n:127.0.0.1:50701_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/96380811796807685-127.0.0.1:50701_-n_0000000000
   [junit4]   2> 303095 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [n:127.0.0.1:50701_    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:50701_
   [junit4]   2> 303096 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [n:127.0.0.1:50701_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 303097 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [n:127.0.0.1:50701_    ] o.a.s.c.Overseer Overseer (id=96380811796807685-127.0.0.1:50701_-n_0000000000) starting
   [junit4]   2> 303098 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [n:127.0.0.1:50701_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 303103 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [n:127.0.0.1:50701_    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 303103 INFO  (OverseerCollectionConfigSetProcessor-96380811796807685-127.0.0.1:50701_-n_0000000000) [n:127.0.0.1:50701_    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 303104 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [n:127.0.0.1:50701_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50701_
   [junit4]   2> 303104 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [n:127.0.0.1:50701_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:50701_
   [junit4]   2> 303104 INFO  (OverseerStateUpdate-96380811796807685-127.0.0.1:50701_-n_0000000000) [n:127.0.0.1:50701_    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 303105 INFO  (zkCallback-500-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [0])
   [junit4]   2> 303105 INFO  (OverseerStateUpdate-96380811796807685-127.0.0.1:50701_-n_0000000000) [n:127.0.0.1:50701_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 303112 INFO  (zkCallback-500-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (1)
   [junit4]   2> 303115 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [n:127.0.0.1:50701_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 303115 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [n:127.0.0.1:50701_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 303115 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [n:127.0.0.1:50701_    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CdcrReplicationDistributedZkTest_C519C4A7A908694E-001/jetty-001/cores
   [junit4]   2> 303116 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [n:127.0.0.1:50701_    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, loadOnStartup=true, configSetProperties=configsetprops.json, collection=tmp_collection, coreNodeName=, dataDir=data/, shard=}
   [junit4]   2> 303116 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [n:127.0.0.1:50701_    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CdcrReplicationDistributedZkTest_C519C4A7A908694E-001/jetty-001/cores/collection1
   [junit4]   2> 303116 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [n:127.0.0.1:50701_    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 303116 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 303116 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 303117 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [n:127.0.0.1:50701_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1
   [junit4]   2> 303117 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [n:127.0.0.1:50701_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 303117 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection   x:collection1] o.a.s.c.ZkController Collection tmp_collection not visible yet, but flagging it so a watch is registered when it becomes visible
   [junit4]   2> 303117 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection   x:collection1] o.a.s.c.c.ZkStateReader Deleting data for [tmp_collection]
   [junit4]   2> 303118 INFO  (OverseerStateUpdate-96380811796807685-127.0.0.1:50701_-n_0000000000) [n:127.0.0.1:50701_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:50701",
   [junit4]   2>   "node_name":"127.0.0.1:50701_",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"tmp_collection",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 303118 INFO  (OverseerStateUpdate-96380811796807685-127.0.0.1:50701_-n_0000000000) [n:127.0.0.1:50701_    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:50701",
   [junit4]   2>   "node_name":"127.0.0.1:50701_",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"tmp_collection",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 303118 INFO  (OverseerStateUpdate-96380811796807685-127.0.0.1:50701_-n_0000000000) [n:127.0.0.1:50701_    ] o.a.s.c.o.ClusterStateMutator building a new cName: tmp_collection
   [junit4]   2> 303118 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 303118 INFO  (OverseerStateUpdate-96380811796807685-127.0.0.1:50701_-n_0000000000) [n:127.0.0.1:50701_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2
   [junit4]   2> 303119 INFO  (zkCallback-500-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 304118 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 304118 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:tmp_collection
   [junit4]   2> 304119 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection   x:collection1] o.a.s.c.ZkController Creating collection in ZooKeeper:tmp_collection
   [junit4]   2> 304119 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection   x:collection1] o.a.s.c.ZkController Looking for collection configName
   [junit4]   2> 304119 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection   x:collection1] o.a.s.c.ZkController Only one config set found in zk - using it:conf1
   [junit4]   2> 304120 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection   x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/tmp_collection
   [junit4]   2> 304121 INFO  (zkCallback-500-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_    ] o.a.s.c.c.ZkStateReader A collections change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections], has occurred - updating...
   [junit4]   2> 304121 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/tmp_collection]
   [junit4]   2> 304122 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection   x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/tmp_collection] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 304122 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CdcrReplicationDistributedZkTest_C519C4A7A908694E-001/jetty-001/cores/collection1'
   [junit4]   2> 304122 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 304122 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 304124 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 304132 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 304134 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.2.0
   [junit4]   2> 304139 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection   x:collection1] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 304145 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 304212 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 304213 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 304213 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 304216 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 304243 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1
   [junit4]   2> 304243 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection tmp_collection
   [junit4]   2> 304244 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 304244 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CdcrReplicationDistributedZkTest_C519C4A7A908694E-001/jetty-001/cores/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CdcrReplicationDistributedZkTest_C519C4A7A908694E-001/jetty-001/cores/collection1/data/]
   [junit4]   2> 304244 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1b94d521
   [junit4]   2> 304244 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CdcrReplicationDistributedZkTest_C519C4A7A908694E-001/jetty-001/cores/collection1/data/snapshot_metadata
   [junit4]   2> 304244 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.s.SolrSnapshotMetaDataManager Loading from snapshot metadata file...
   [junit4]   2> 304245 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CdcrReplicationDistributedZkTest_C519C4A7A908694E-001/jetty-001/cores/collection1/data
   [junit4]   2> 304245 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CdcrReplicationDistributedZkTest_C519C4A7A908694E-001/jetty-001/cores/collection1/data/index/
   [junit4]   2> 304245 WARN  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CdcrReplicationDistributedZkTest_C519C4A7A908694E-001/jetty-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 304245 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CdcrReplicationDistributedZkTest_C519C4A7A908694E-001/jetty-001/cores/collection1/data/index
   [junit4]   2> 304246 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=575335063, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
   [junit4]   2> 304246 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=RawDirectoryWrapper(RAMDirectory@27214a89 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@63d07e4),segFN=segments_1,generation=1}
   [junit4]   2> 304246 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 304247 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CdcrReplicationDistributedZkTest_C519C4A7A908694E-001/jetty-001/cores/collection1/data/
   [junit4]   2> 304247 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CdcrReplicationDistributedZkTest_C519C4A7A908694E-001/jetty-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 304249 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "cdcr-processor-chain"
   [junit4]   2> 304249 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 304250 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 304251 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 304251 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 304252 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 304256 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/mbeans,standard,/update/json/docs,/admin/luke,/export,/get,/admin/properties,/update/json,/admin/threads,/update/csv,/sql,/graph,/admin/segments,/admin/system,/replication,/config,/stream,/schema,/admin/plugins,/admin/logging,/admin/ping,/cdcr,/update,/admin/file,/terms
   [junit4]   2> 304256 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 304256 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.CdcrUpdateLog
   [junit4]   2> 304256 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 304257 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 304257 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 304257 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=14, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.4610309097504345]
   [junit4]   2> 304257 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=RawDirectoryWrapper(RAMDirectory@27214a89 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@63d07e4),segFN=segments_1,generation=1}
   [junit4]   2> 304257 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 304257 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@5b0b6264[collection1] main]
   [junit4]   2> 304258 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/tmp_collection]
   [junit4]   2> 304258 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/tmp_collection] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 304258 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 304258 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 304258 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 304258 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 304259 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 304260 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 304260 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 304260 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 304260 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/tmp_collection/cdcr/state
   [junit4]   2> 304263 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.h.CdcrBufferStateManager Created znode /collections/tmp_collection/cdcr/state/buffer
   [junit4]   2> 304264 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.h.CdcrProcessStateManager Created znode /collections/tmp_collection/cdcr/state/process
   [junit4]   2> 304265 INFO  (searcherExecutor-1500-thread-1-processing-n:127.0.0.1:50701_ x:collection1 s:shard2 c:tmp_collection r:core_node1) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@5b0b6264[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 304265 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 304265 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 304265 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 304265 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1542092990249435136
   [junit4]   2> 304267 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 2.0ms to seed version buckets with highest version 1542092990249435136
   [junit4]   2> 304267 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 304267 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 304268 INFO  (coreZkRegister-1493-thread-1-processing-n:127.0.0.1:50701_ x:collection1 s:shard2 c:tmp_collection r:core_node1) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:https://127.0.0.1:50701 collection:tmp_collection shard:shard2
   [junit4]   2> 304269 INFO  (coreZkRegister-1493-thread-1-processing-n:127.0.0.1:50701_ x:collection1 s:shard2 c:tmp_collection r:core_node1) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/tmp_collection/leader_elect/shard2/election
   [junit4]   2> 304278 INFO  (coreZkRegister-1493-thread-1-processing-n:127.0.0.1:50701_ x:collection1 s:shard2 c:tmp_collection r:core_node1) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/tmp_collection/leader_elect/shard2/election/96380811796807685-core_node1-n_0000000000
   [junit4]   2> 304278 INFO  (coreZkRegister-1493-thread-1-processing-n:127.0.0.1:50701_ x:collection1 s:shard2 c:tmp_collection r:core_node1) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard=shard2 and weAreReplacement=false and leaderVoteWait=10000
   [junit4]   2> 304280 INFO  (coreZkRegister-1493-thread-1-processing-n:127.0.0.1:50701_ x:collection1 s:shard2 c:tmp_collection r:core_node1) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 304280 INFO  (coreZkRegister-1493-thread-1-processing-n:127.0.0.1:50701_ x:collection1 s:shard2 c:tmp_collection r:core_node1) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 304280 INFO  (coreZkRegister-1493-thread-1-processing-n:127.0.0.1:50701_ x:collection1 s:shard2 c:tmp_collection r:core_node1) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:50701/collection1/
   [junit4]   2> 304280 INFO  (coreZkRegister-1493-thread-1-processing-n:127.0.0.1:50701_ x:collection1 s:shard2 c:tmp_collection r:core_node1) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 304280 INFO  (coreZkRegister-1493-thread-1-processing-n:127.0.0.1:50701_ x:collection1 s:shard2 c:tmp_collection r:core_node1) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.SyncStrategy https://127.0.0.1:50701/collection1/ has no replicas
   [junit4]   2> 304281 INFO  (OverseerStateUpdate-96380811796807685-127.0.0.1:50701_-n_0000000000) [n:127.0.0.1:50701_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard2",
   [junit4]   2>   "collection":"tmp_collection"} current state version: 1
   [junit4]   2> 304281 INFO  (coreZkRegister-1493-thread-1-processing-n:127.0.0.1:50701_ x:collection1 s:shard2 c:tmp_collection r:core_node1) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/tmp_collection/leaders/shard2
   [junit4]   2> 304283 INFO  (coreZkRegister-1493-thread-1-processing-n:127.0.0.1:50701_ x:collection1 s:shard2 c:tmp_collection r:core_node1) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/tmp_collection/leaders/shard2/leader after winning as /collections/tmp_collection/leader_elect/shard2/election/96380811796807685-core_node1-n_0000000000
   [junit4]   2> 304283 INFO  (zkCallback-500-thread-2-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_    ] o.a.s.h.CdcrLeaderStateManager Received new leader state @ tmp_collection:shard2
   [junit4]   2> 304284 INFO  (coreZkRegister-1493-thread-1-processing-n:127.0.0.1:50701_ x:collection1 s:shard2 c:tmp_collection r:core_node1) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:50701/collection1/ shard2
   [junit4]   2> 304285 INFO  (OverseerStateUpdate-96380811796807685-127.0.0.1:50701_-n_0000000000) [n:127.0.0.1:50701_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard2",
   [junit4]   2>   "collection":"tmp_collection",
   [junit4]   2>   "base_url":"https://127.0.0.1:50701",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 1
   [junit4]   2> 304386 INFO  (zkCallback-500-thread-2-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 304435 INFO  (coreZkRegister-1493-thread-1-processing-n:127.0.0.1:50701_ x:collection1 s:shard2 c:tmp_collection r:core_node1) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.ZkController We are https://127.0.0.1:50701/collection1/ and leader is https://127.0.0.1:50701/collection1/
   [junit4]   2> 304435 INFO  (coreZkRegister-1493-thread-1-processing-n:127.0.0.1:50701_ x:collection1 s:shard2 c:tmp_collection r:core_node1) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:50701
   [junit4]   2> 304436 INFO  (coreZkRegister-1493-thread-1-processing-n:127.0.0.1:50701_ x:collection1 s:shard2 c:tmp_collection r:core_node1) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 304436 INFO  (coreZkRegister-1493-thread-1-processing-n:127.0.0.1:50701_ x:collection1 s:shard2 c:tmp_collection r:core_node1) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 304436 INFO  (coreZkRegister-1493-thread-1-processing-n:127.0.0.1:50701_ x:collection1 s:shard2 c:tmp_collection r:core_node1) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 304436 INFO  (coreZkRegister-1493-thread-1-processing-n:127.0.0.1:50701_ x:collection1 s:shard2 c:tmp_collection r:core_node1) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Checking legacy cluster state for collection tmp_collection
   [junit4]   2> 304437 INFO  (OverseerStateUpdate-96380811796807685-127.0.0.1:50701_-n_0000000000) [n:127.0.0.1:50701_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:50701",
   [junit4]   2>   "node_name":"127.0.0.1:50701_",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard2",
   [junit4]   2>   "collection":"tmp_collection",
   [junit4]   2>   "operation":"state"} current state version: 2
   [junit4]   2> 304437 INFO  (OverseerStateUpdate-96380811796807685-127.0.0.1:50701_-n_0000000000) [n:127.0.0.1:50701_    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:50701",
   [junit4]   2>   "node_name":"127.0.0.1:50701_",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard2",
   [junit4]   2>   "collection":"tmp_collection",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 304541 INFO  (zkCallback-500-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 304692 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CdcrReplicationDistributedZkTest_C519C4A7A908694E-001/jetty-002/cores/collection1
   [junit4]   2> 304694 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.e.j.s.Server jetty-9.3.8.v20160314
   [junit4]   2> 304696 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@2e703e69{/,null,AVAILABLE}
   [junit4]   2> 304699 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.e.j.s.ServerConnector Started ServerConnector@2eb367ed{SSL,[ssl, http/1.1]}{127.0.0.1:47996}
   [junit4]   2> 304699 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.e.j.s.Server Started @306852ms
   [junit4]   2> 304699 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {shards=shard2, hostContext=/, hostPort=47996, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CdcrReplicationDistributedZkTest_C519C4A7A908694E-001/jetty-002/cores}
   [junit4]   2> 304700 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@73d16e93
   [junit4]   2> 304700 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CdcrReplicationDistributedZkTest_C519C4A7A908694E-001/jetty-002'
   [junit4]   2> 304700 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 304700 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 304702 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 304702 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 304704 INFO  (zkCallback-502-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@59ba42bb name:ZooKeeperConnection Watcher:127.0.0.1:41903/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 304704 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 304704 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 304705 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 304705 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CdcrReplicationDistributedZkTest_C519C4A7A908694E-001/jetty-002/solr.xml
   [junit4]   2> 304709 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CdcrReplicationDistributedZkTest_C519C4A7A908694E-001/jetty-002/cores
   [junit4]   2> 304709 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.CoreContainer New CoreContainer 900869275
   [junit4]   2> 304709 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CdcrReplicationDistributedZkTest_C519C4A7A908694E-001/jetty-002]
   [junit4]   2> 304709 WARN  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CdcrReplicationDistributedZkTest_C519C4A7A908694E-001/jetty-002/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CdcrReplicationDistributedZkTest_C519C4A7A908694E-001/jetty-002/lib
   [junit4]   2> 304710 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,connectionsEvictorSleepDelay : 5000,maxConnectionIdleTime : 40000,
   [junit4]   2> 304711 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 304711 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 304711 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 304711 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:41903/solr
   [junit4]   2> 304711 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 304711 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 304716 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 304718 INFO  (zkCallback-505-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4b8152b9 name:ZooKeeperConnection Watcher:127.0.0.1:41903 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 304718 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 304718 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 304719 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [n:127.0.0.1:47996_    ] o.a.s.c.ZkController Added new OnReconnect listener org.apache.solr.cloud.ZkController$$Lambda$10/276052250@48471094
   [junit4]   2> 304721 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [n:127.0.0.1:47996_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 304722 INFO  (zkCallback-506-thread-1-processing-n:127.0.0.1:47996_) [n:127.0.0.1:47996_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6b1f9a2c name:ZooKeeperConnection Watcher:127.0.0.1:41903/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 304722 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [n:127.0.0.1:47996_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 304735 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [n:127.0.0.1:47996_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 304736 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [n:127.0.0.1:47996_    ] o.a.s.c.c.ZkStateReader Loaded cluster properties: {urlScheme=https}
   [junit4]   2> 304736 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [n:127.0.0.1:47996_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 304740 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [n:127.0.0.1:47996_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 304740 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [n:127.0.0.1:47996_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/96380811796807688-127.0.0.1:47996_-n_0000000001
   [junit4]   2> 304741 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [n:127.0.0.1:47996_    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/96380811796807685-127.0.0.1:50701_-n_0000000000 to know if I could be the leader
   [junit4]   2> 304741 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [n:127.0.0.1:47996_    ] o.a.s.c.ZkController Publish node=127.0.0.1:47996_ as DOWN
   [junit4]   2> 304743 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [n:127.0.0.1:47996_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:47996_
   [junit4]   2> 304743 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [n:127.0.0.1:47996_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:47996_
   [junit4]   2> 304743 INFO  (OverseerStateUpdate-96380811796807685-127.0.0.1:50701_-n_0000000000) [n:127.0.0.1:50701_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"downnode",
   [junit4]   2>   "node_name":"127.0.0.1:47996_"} current state version: 3
   [junit4]   2> 304743 INFO  (OverseerStateUpdate-96380811796807685-127.0.0.1:50701_-n_0000000000) [n:127.0.0.1:50701_    ] o.a.s.c.o.NodeMutator DownNode state invoked for node: 127.0.0.1:47996_
   [junit4]   2> 304744 INFO  (zkCallback-500-thread-3-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 304744 INFO  (zkCallback-506-thread-1-processing-n:127.0.0.1:47996_) [n:127.0.0.1:47996_    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 304745 INFO  (zkCallback-500-thread-3-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 304749 INFO  (zkCallback-506-thread-1-processing-n:127.0.0.1:47996_) [n:127.0.0.1:47996_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 304755 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [n:127.0.0.1:47996_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 304755 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [n:127.0.0.1:47996_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 304756 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [n:127.0.0.1:47996_    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CdcrReplicationDistributedZkTest_C519C4A7A908694E-001/jetty-002/cores
   [junit4]   2> 304757 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [n:127.0.0.1:47996_    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, loadOnStartup=true, configSetProperties=configsetprops.json, collection=tmp_collection, coreNodeName=, dataDir=data/, shard=}
   [junit4]   2> 304757 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [n:127.0.0.1:47996_    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CdcrReplicationDistributedZkTest_C519C4A7A908694E-001/jetty-002/cores/collection1
   [junit4]   2> 304757 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [n:127.0.0.1:47996_    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 304758 INFO  (coreLoadExecutor-1511-thread-1-processing-n:127.0.0.1:47996_) [n:127.0.0.1:47996_ c:tmp_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 304758 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [n:127.0.0.1:47996_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1
   [junit4]   2> 304758 INFO  (coreLoadExecutor-1511-thread-1-processing-n:127.0.0.1:47996_) [n:127.0.0.1:47996_ c:tmp_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 304758 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [n:127.0.0.1:47996_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 304759 INFO  (coreLoadExecutor-1511-thread-1-processing-n:127.0.0.1:47996_) [n:127.0.0.1:47996_ c:tmp_collection   x:collection1] o.a.s.c.ZkController Registering watch for collection tmp_collection
   [junit4]   2> 304759 INFO  (coreLoadExecutor-1511-thread-1-processing-n:127.0.0.1:47996_) [n:127.0.0.1:47996_ c:tmp_collection   x:collection1] o.a.s.c.c.ZkStateReader Deleting data for [tmp_collection]
   [junit4]   2> 304759 INFO  (OverseerStateUpdate-96380811796807685-127.0.0.1:50701_-n_0000000000) [n:127.0.0.1:50701_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:47996",
   [junit4]   2>   "node_name":"127.0.0.1:47996_",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"tmp_collection",
   [junit4]   2>   "operation":"state"} current state version: 3
   [junit4]   2> 304760 INFO  (OverseerStateUpdate-96380811796807685-127.0.0.1:50701_-n_000

[...truncated too long message...]

exception-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-journal-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-normalization-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-operational-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-referral-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-schema-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-subtree-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-trigger-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-jdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-jdbm1-2.0.0-M2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-kerberos-codec-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-ldif-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-mavibot-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-protocol-kerberos-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-protocol-ldap-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-protocol-shared-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-xdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/api-all-1.0.0-M20.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/bcprov-jdk15-1.45.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/cglib-nodep-2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/commons-collections-3.2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/commons-math3-3.4.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/easymock-3.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/ehcache-core-2.4.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/hadoop-common-2.7.2-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/hadoop-hdfs-2.7.2-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/hadoop-minikdc-2.7.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/htrace-core-3.2.0-incubating.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jackson-annotations-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jackson-core-asl-1.9.13.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jackson-databind-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jackson-mapper-asl-1.9.13.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jersey-core-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jersey-server-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jetty-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jetty-util-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/mina-core-2.0.0-M5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/mockito-core-1.9.5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/netty-all-4.0.36.Final.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/objenesis-1.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/presto-parser-0.122.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/slice-0.10.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/analysis/icu/lucene-analyzers-icu-6.2.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/contrib/solr-analysis-extras/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/queryparser/classes/test:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/contrib/analysis-extras/lib/icu4j-56.1.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-launcher.jar:/x1/jenkins/.ant/lib/ivy-2.3.0.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-antlr.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-regexp.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-resolver.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-net.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jmf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-oro.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-xalan2.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-logging.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jsch.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit4.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-javamail.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jai.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jdepend.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bsf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-swing.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-netrexx.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bcel.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-testutil.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-log4j.jar:/x1/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.8/lib/tools.jar:/x1/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.3.4.jar com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/temp/junit4-J2-20160808_110251_850.events @/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/temp/junit4-J2-20160808_110251_850.suites -stdin
   [junit4] ERROR: JVM J2 ended with an exception: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1501)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:121)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:955)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:952)
   [junit4] 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4] 	at java.lang.Thread.run(Thread.java:745)

BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:771: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:707: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/common-build.xml:536: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/common-build.xml:1443: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/common-build.xml:984: At least one slave process threw an exception, first: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.

Total time: 295 minutes 34 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any




[JENKINS] Lucene-Solr-NightlyTests-6.x - Build # 139 - Failure

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-6.x/139/

3 tests failed.
FAILED:  org.apache.solr.cloud.LeaderFailoverAfterPartitionTest.test

Error Message:
No live SolrServers available to handle this request:[http://127.0.0.1:55214/hd_om/c8n_1x3_lf_shard1_replica2]

Stack Trace:
org.apache.solr.client.solrj.impl.CloudSolrClient$RouteException: No live SolrServers available to handle this request:[http://127.0.0.1:55214/hd_om/c8n_1x3_lf_shard1_replica2]
	at __randomizedtesting.SeedInfo.seed([4C17D787E792729C:C443E85D496E1F64]:0)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.directUpdate(CloudSolrClient.java:760)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1172)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1061)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:997)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
	at org.apache.solr.cloud.HttpPartitionTest.sendDoc(HttpPartitionTest.java:592)
	at org.apache.solr.cloud.HttpPartitionTest.sendDoc(HttpPartitionTest.java:578)
	at org.apache.solr.cloud.LeaderFailoverAfterPartitionTest.testRf3WithLeaderFailover(LeaderFailoverAfterPartitionTest.java:174)
	at org.apache.solr.cloud.LeaderFailoverAfterPartitionTest.test(LeaderFailoverAfterPartitionTest.java:55)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:55214/hd_om/c8n_1x3_lf_shard1_replica2]
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:383)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.lambda$directUpdate$0(CloudSolrClient.java:735)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	... 1 more
Caused by: org.apache.solr.client.solrj.SolrServerException: Server refused connection at: http://127.0.0.1:55214/hd_om/c8n_1x3_lf_shard1_replica2
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:599)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:261)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:250)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:403)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:356)
	... 6 more
Caused by: java.net.ConnectException: Connection refused
	at java.net.PlainSocketImpl.socketConnect(Native Method)
	at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
	at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
	at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
	at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
	at java.net.Socket.connect(Socket.java:589)
	at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:117)
	at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:177)
	at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:304)
	at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:611)
	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:446)
	at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:497)
	... 10 more


FAILED:  org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.test

Error Message:
Timeout occured while waiting response from server at: http://127.0.0.1:48129

Stack Trace:
org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:48129
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:603)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:261)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:250)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.makeRequest(CollectionsAPIDistributedZkTest.java:399)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testErrorHandling(CollectionsAPIDistributedZkTest.java:515)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test(CollectionsAPIDistributedZkTest.java:179)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.net.SocketTimeoutException: Read timed out
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
	at java.net.SocketInputStream.read(SocketInputStream.java:170)
	at java.net.SocketInputStream.read(SocketInputStream.java:141)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:160)
	at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:84)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:273)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
	at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
	at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)
	at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)
	at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197)
	at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)
	at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)
	at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685)
	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487)
	at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:497)
	... 47 more


FAILED:  org.apache.solr.handler.TestReqParamsAPI.test

Error Message:
Could not get expected value  'CY val' for path 'response/params/y/c' full output: {   "responseHeader":{     "status":0,     "QTime":0},   "response":{     "znodeVersion":0,     "params":{"x":{         "a":"A val",         "b":"B val",         "":{"v":0}}}}},  from server:  https://127.0.0.1:37395/collection1

Stack Trace:
java.lang.AssertionError: Could not get expected value  'CY val' for path 'response/params/y/c' full output: {
  "responseHeader":{
    "status":0,
    "QTime":0},
  "response":{
    "znodeVersion":0,
    "params":{"x":{
        "a":"A val",
        "b":"B val",
        "":{"v":0}}}}},  from server:  https://127.0.0.1:37395/collection1
	at __randomizedtesting.SeedInfo.seed([4C17D787E792729C:C443E85D496E1F64]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.apache.solr.core.TestSolrConfigHandler.testForResponseElement(TestSolrConfigHandler.java:481)
	at org.apache.solr.handler.TestReqParamsAPI.testReqParams(TestReqParamsAPI.java:159)
	at org.apache.solr.handler.TestReqParamsAPI.test(TestReqParamsAPI.java:61)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 11895 lines...]
   [junit4] JVM J1: stdout was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/temp/junit4-J1-20160807_105930_686.sysout
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] java.lang.OutOfMemoryError: GC overhead limit exceeded
   [junit4] Dumping heap to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/heapdumps/java_pid21736.hprof ...
   [junit4] Heap dump file created [712416286 bytes in 6.233 secs]
   [junit4] <<< JVM J1: EOF ----

   [junit4] JVM J1: stderr was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/temp/junit4-J1-20160807_105930_686.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] WARN: Unhandled exception in event serialization. -> java.lang.OutOfMemoryError: GC overhead limit exceeded
   [junit4] <<< JVM J1: EOF ----

[...truncated 46 lines...]
   [junit4] Suite: org.apache.solr.cloud.LeaderFailoverAfterPartitionTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_4C17D787E792729C-001/init-core-data-001
   [junit4]   2> 1995750 INFO  (SUITE-LeaderFailoverAfterPartitionTest-seed#[4C17D787E792729C]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=https://issues.apache.org/jira/browse/SOLR-5776)
   [junit4]   2> 1995750 INFO  (SUITE-LeaderFailoverAfterPartitionTest-seed#[4C17D787E792729C]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /hd_om/
   [junit4]   2> 1995754 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 1995755 INFO  (Thread-19798) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1995755 INFO  (Thread-19798) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 1995855 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.ZkTestServer start zk server on port:36677
   [junit4]   2> 1995855 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1995855 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1995858 INFO  (zkCallback-1867-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@674feee7 name:ZooKeeperConnection Watcher:127.0.0.1:36677 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1995859 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1995859 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1995859 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 1995861 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1995865 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1995865 INFO  (zkCallback-1868-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1eb7aec8 name:ZooKeeperConnection Watcher:127.0.0.1:36677/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1995865 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1995865 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1995865 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 1995866 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 1995868 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 1995868 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 1995870 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 1995870 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 1995872 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 1995872 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 1995873 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1995874 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1995875 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 1995875 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 1995876 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 1995876 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 1995877 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 1995877 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 1995878 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 1995878 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 1995879 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1995880 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1995881 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1995881 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1995882 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 1995882 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 1995883 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 1995883 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 1995952 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_4C17D787E792729C-001/control-001/cores/collection1
   [junit4]   2> 1995954 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.e.j.s.Server jetty-9.3.8.v20160314
   [junit4]   2> 1995956 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5f27f9f{/hd_om,null,AVAILABLE}
   [junit4]   2> 1995964 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.e.j.s.ServerConnector Started ServerConnector@62d198b4{HTTP/1.1,[http/1.1]}{127.0.0.1:53562}
   [junit4]   2> 1995964 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.e.j.s.Server Started @1998175ms
   [junit4]   2> 1995964 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_4C17D787E792729C-001/tempDir-001/control/data, hostContext=/hd_om, hostPort=52303, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_4C17D787E792729C-001/control-001/cores}
   [junit4]   2> 1995964 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@73d16e93
   [junit4]   2> 1995964 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_4C17D787E792729C-001/control-001'
   [junit4]   2> 1995964 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1995964 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1995965 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1995969 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1995970 INFO  (zkCallback-1869-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6d70da7d name:ZooKeeperConnection Watcher:127.0.0.1:36677/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1995970 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1995970 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1995971 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1995971 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_4C17D787E792729C-001/control-001/solr.xml
   [junit4]   2> 1995975 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_4C17D787E792729C-001/control-001/cores
   [junit4]   2> 1995975 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.CoreContainer New CoreContainer 1477612659
   [junit4]   2> 1995975 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_4C17D787E792729C-001/control-001]
   [junit4]   2> 1995975 WARN  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_4C17D787E792729C-001/control-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_4C17D787E792729C-001/control-001/lib
   [junit4]   2> 1995975 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,connectionsEvictorSleepDelay : 5000,maxConnectionIdleTime : 40000,
   [junit4]   2> 1995976 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1995976 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1995976 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1995976 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:36677/solr
   [junit4]   2> 1995976 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1995977 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1995977 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1995978 INFO  (zkCallback-1872-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@32410b62 name:ZooKeeperConnection Watcher:127.0.0.1:36677 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1995979 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1995979 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1995980 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [n:127.0.0.1:52303_hd_om    ] o.a.s.c.ZkController Added new OnReconnect listener org.apache.solr.cloud.ZkController$$Lambda$96/145199802@392e3620
   [junit4]   2> 1995981 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [n:127.0.0.1:52303_hd_om    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1995981 INFO  (zkCallback-1873-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@28c3d5d3 name:ZooKeeperConnection Watcher:127.0.0.1:36677/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1995981 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [n:127.0.0.1:52303_hd_om    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1995982 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [n:127.0.0.1:52303_hd_om    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 1995983 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [n:127.0.0.1:52303_hd_om    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 1995985 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [n:127.0.0.1:52303_hd_om    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 1995986 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [n:127.0.0.1:52303_hd_om    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 1995987 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [n:127.0.0.1:52303_hd_om    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 1995987 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [n:127.0.0.1:52303_hd_om    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 1995988 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [n:127.0.0.1:52303_hd_om    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 1995989 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [n:127.0.0.1:52303_hd_om    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 1995989 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [n:127.0.0.1:52303_hd_om    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 1995990 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [n:127.0.0.1:52303_hd_om    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1995990 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [n:127.0.0.1:52303_hd_om    ] o.a.s.c.c.ZkStateReader Loaded empty cluster properties
   [junit4]   2> 1995990 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [n:127.0.0.1:52303_hd_om    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (0)
   [junit4]   2> 1995992 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [n:127.0.0.1:52303_hd_om    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 1995992 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [n:127.0.0.1:52303_hd_om    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 1995993 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [n:127.0.0.1:52303_hd_om    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1995993 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [n:127.0.0.1:52303_hd_om    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/96375247254716420-127.0.0.1:52303_hd_om-n_0000000000
   [junit4]   2> 1995993 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [n:127.0.0.1:52303_hd_om    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:52303_hd_om
   [junit4]   2> 1995993 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [n:127.0.0.1:52303_hd_om    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 1995994 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [n:127.0.0.1:52303_hd_om    ] o.a.s.c.Overseer Overseer (id=96375247254716420-127.0.0.1:52303_hd_om-n_0000000000) starting
   [junit4]   2> 1995995 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [n:127.0.0.1:52303_hd_om    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 1995997 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [n:127.0.0.1:52303_hd_om    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 1995998 INFO  (OverseerStateUpdate-96375247254716420-127.0.0.1:52303_hd_om-n_0000000000) [n:127.0.0.1:52303_hd_om    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 1995998 INFO  (OverseerCollectionConfigSetProcessor-96375247254716420-127.0.0.1:52303_hd_om-n_0000000000) [n:127.0.0.1:52303_hd_om    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 1996002 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [n:127.0.0.1:52303_hd_om    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:52303_hd_om
   [junit4]   2> 1996002 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [n:127.0.0.1:52303_hd_om    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:52303_hd_om
   [junit4]   2> 1996002 INFO  (OverseerStateUpdate-96375247254716420-127.0.0.1:52303_hd_om-n_0000000000) [n:127.0.0.1:52303_hd_om    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (0)
   [junit4]   2> 1996002 INFO  (zkCallback-1873-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [0])
   [junit4]   2> 1996005 INFO  (zkCallback-1873-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1996015 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [n:127.0.0.1:52303_hd_om    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1996015 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [n:127.0.0.1:52303_hd_om    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1996015 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [n:127.0.0.1:52303_hd_om    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_4C17D787E792729C-001/control-001/cores
   [junit4]   2> 1996016 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [n:127.0.0.1:52303_hd_om    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, loadOnStartup=true, configSetProperties=configsetprops.json, collection=control_collection, coreNodeName=, dataDir=data/, shard=}
   [junit4]   2> 1996016 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [n:127.0.0.1:52303_hd_om    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_4C17D787E792729C-001/control-001/cores/collection1
   [junit4]   2> 1996016 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [n:127.0.0.1:52303_hd_om    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 1996016 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 1996016 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1996017 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [n:127.0.0.1:52303_hd_om    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2
   [junit4]   2> 1996017 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [n:127.0.0.1:52303_hd_om    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1996017 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection   x:collection1] o.a.s.c.ZkController Collection control_collection not visible yet, but flagging it so a watch is registered when it becomes visible
   [junit4]   2> 1996017 INFO  (OverseerStateUpdate-96375247254716420-127.0.0.1:52303_hd_om-n_0000000000) [n:127.0.0.1:52303_hd_om    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:52303/hd_om",
   [junit4]   2>   "node_name":"127.0.0.1:52303_hd_om",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 1996018 INFO  (OverseerStateUpdate-96375247254716420-127.0.0.1:52303_hd_om-n_0000000000) [n:127.0.0.1:52303_hd_om    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:52303/hd_om",
   [junit4]   2>   "node_name":"127.0.0.1:52303_hd_om",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 1996018 INFO  (OverseerStateUpdate-96375247254716420-127.0.0.1:52303_hd_om-n_0000000000) [n:127.0.0.1:52303_hd_om    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 1996019 INFO  (OverseerStateUpdate-96375247254716420-127.0.0.1:52303_hd_om-n_0000000000) [n:127.0.0.1:52303_hd_om    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 1996019 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Deleting data for [control_collection]
   [junit4]   2> 1996020 INFO  (zkCallback-1873-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1996025 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 1996025 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1996025 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 1996026 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 1996026 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/control_collection]
   [junit4]   2> 1996026 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/control_collection] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 1996026 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_4C17D787E792729C-001/control-001/cores/collection1'
   [junit4]   2> 1996026 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1996027 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1996028 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 1996033 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 1996037 WARN  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection   x:collection1] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
   [junit4]   2> 1996037 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.2.0
   [junit4]   2> 1996044 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection   x:collection1] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1996048 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1996125 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 1996131 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 1996132 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 1996143 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1996145 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1996146 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1996147 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1996148 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2
   [junit4]   2> 1996148 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 1996148 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 1996148 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_4C17D787E792729C-001/control-001/cores/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_4C17D787E792729C-001/control-001/cores/collection1/data/]
   [junit4]   2> 1996148 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@18577059
   [junit4]   2> 1996149 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_4C17D787E792729C-001/control-001/cores/collection1/data/snapshot_metadata
   [junit4]   2> 1996149 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.s.SolrSnapshotMetaDataManager Loading from snapshot metadata file...
   [junit4]   2> 1996149 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_4C17D787E792729C-001/control-001/cores/collection1/data
   [junit4]   2> 1996149 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_4C17D787E792729C-001/control-001/cores/collection1/data/index/
   [junit4]   2> 1996149 WARN  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_4C17D787E792729C-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 1996149 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_4C17D787E792729C-001/control-001/cores/collection1/data/index
   [junit4]   2> 1996150 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=46, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.4530217309050658]
   [junit4]   2> 1996150 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=RawDirectoryWrapper(RAMDirectory@4a4578ae lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2bb91bf0),segFN=segments_1,generation=1}
   [junit4]   2> 1996150 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 1996150 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_4C17D787E792729C-001/control-001/cores/collection1/data/
   [junit4]   2> 1996150 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_4C17D787E792729C-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 1996154 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 1996155 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 1996155 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 1996155 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1996155 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1996155 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 1996155 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1996155 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1996155 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 1996155 WARN  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
   [junit4]   2> 1996156 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1996157 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1996157 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1996158 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1996160 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/mbeans,standard,/update/json/docs,/admin/luke,/export,/get,/admin/properties,/update/json,/admin/threads,/dump,/update/csv,/sql,/graph,/admin/segments,/admin/system,/replication,/config,/stream,/schema,/admin/plugins,/admin/logging,/admin/ping,/update,/admin/file,/terms
   [junit4]   2> 1996160 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 1996160 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1996160 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1996161 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1996161 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1996161 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=24, maxMergeAtOnceExplicit=20, maxMergedSegmentMB=0.3271484375, floorSegmentMB=0.27734375, forceMergeDeletesPctAllowed=2.176793134409356, segmentsPerTier=42.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 1996161 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=RawDirectoryWrapper(RAMDirectory@4a4578ae lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2bb91bf0),segFN=segments_1,generation=1}
   [junit4]   2> 1996161 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 1996161 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@66463c78[collection1] main]
   [junit4]   2> 1996161 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/control_collection]
   [junit4]   2> 1996162 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/control_collection] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 1996162 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 1996162 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1996162 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 1996163 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1996163 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 1996163 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1996163 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 1996163 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1996164 INFO  (searcherExecutor-5063-thread-1-processing-n:127.0.0.1:52303_hd_om x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@66463c78[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1996164 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 1996164 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 1996164 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 1996164 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1542003956503281664
   [junit4]   2> 1996168 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 3.0ms to seed version buckets with highest version 1542003956503281664
   [junit4]   2> 1996168 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 1996169 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 1996169 INFO  (coreZkRegister-5056-thread-1-processing-n:127.0.0.1:52303_hd_om x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:52303/hd_om collection:control_collection shard:shard1
   [junit4]   2> 1996170 INFO  (coreZkRegister-5056-thread-1-processing-n:127.0.0.1:52303_hd_om x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 1996171 INFO  (coreZkRegister-5056-thread-1-processing-n:127.0.0.1:52303_hd_om x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/96375247254716420-core_node1-n_0000000000
   [junit4]   2> 1996171 INFO  (coreZkRegister-5056-thread-1-processing-n:127.0.0.1:52303_hd_om x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard=shard1 and weAreReplacement=false and leaderVoteWait=10000
   [junit4]   2> 1996172 INFO  (coreZkRegister-5056-thread-1-processing-n:127.0.0.1:52303_hd_om x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1996172 INFO  (coreZkRegister-5056-thread-1-processing-n:127.0.0.1:52303_hd_om x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1996172 INFO  (coreZkRegister-5056-thread-1-processing-n:127.0.0.1:52303_hd_om x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:52303/hd_om/collection1/
   [junit4]   2> 1996172 INFO  (OverseerStateUpdate-96375247254716420-127.0.0.1:52303_hd_om-n_0000000000) [n:127.0.0.1:52303_hd_om    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 1996172 INFO  (coreZkRegister-5056-thread-1-processing-n:127.0.0.1:52303_hd_om x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 1996173 INFO  (coreZkRegister-5056-thread-1-processing-n:127.0.0.1:52303_hd_om x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:52303/hd_om/collection1/ has no replicas
   [junit4]   2> 1996173 INFO  (coreZkRegister-5056-thread-1-processing-n:127.0.0.1:52303_hd_om x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 1996175 INFO  (coreZkRegister-5056-thread-1-processing-n:127.0.0.1:52303_hd_om x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/control_collection/leaders/shard1/leader after winning as /collections/control_collection/leader_elect/shard1/election/96375247254716420-core_node1-n_0000000000
   [junit4]   2> 1996176 INFO  (coreZkRegister-5056-thread-1-processing-n:127.0.0.1:52303_hd_om x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:52303/hd_om/collection1/ shard1
   [junit4]   2> 1996181 INFO  (OverseerStateUpdate-96375247254716420-127.0.0.1:52303_hd_om-n_0000000000) [n:127.0.0.1:52303_hd_om    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:52303/hd_om",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 1
   [junit4]   2> 1996283 INFO  (zkCallback-1873-thread-3-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1996327 INFO  (coreZkRegister-5056-thread-1-processing-n:127.0.0.1:52303_hd_om x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:52303/hd_om/collection1/ and leader is http://127.0.0.1:52303/hd_om/collection1/
   [junit4]   2> 1996327 INFO  (coreZkRegister-5056-thread-1-processing-n:127.0.0.1:52303_hd_om x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:52303/hd_om
   [junit4]   2> 1996328 INFO  (coreZkRegister-5056-thread-1-processing-n:127.0.0.1:52303_hd_om x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1996328 INFO  (coreZkRegister-5056-thread-1-processing-n:127.0.0.1:52303_hd_om x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 1996328 INFO  (coreZkRegister-5056-thread-1-processing-n:127.0.0.1:52303_hd_om x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1996329 INFO  (coreZkRegister-5056-thread-1-processing-n:127.0.0.1:52303_hd_om x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Checking legacy cluster state for collection control_collection
   [junit4]   2> 1996330 INFO  (OverseerStateUpdate-96375247254716420-127.0.0.1:52303_hd_om-n_0000000000) [n:127.0.0.1:52303_hd_om    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:52303/hd_om",
   [junit4]   2>   "node_name":"127.0.0.1:52303_hd_om",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"} current state version: 2
   [junit4]   2> 1996330 INFO  (OverseerStateUpdate-96375247254716420-127.0.0.1:52303_hd_om-n_0000000000) [n:127.0.0.1:52303_hd_om    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:52303/hd_om",
   [junit4]   2>   "node_name":"127.0.0.1:52303_hd_om",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 1996432 INFO  (zkCallback-1873-thread-3-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1996520 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1996520 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1996521 INFO  (zkCallback-1877-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@33f25bb7 name:ZooKeeperConnection Watcher:127.0.0.1:36677/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1996521 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1996522 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1996522 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1996522 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.c.ZkStateReader Loaded empty cluster properties
   [junit4]   2> 1996525 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1996526 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 1996526 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with stateFormat=2
   [junit4]   2> 1996526 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1996526 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1996527 INFO  (zkCallback-1878-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2dd1006 name:ZooKeeperConnection Watcher:127.0.0.1:36677/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1996527 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1996528 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1996529 INFO  (OverseerStateUpdate-96375247254716420-127.0.0.1:52303_hd_om-n_0000000000) [n:127.0.0.1:52303_hd_om    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"create",
   [junit4]   2>   "name":"collection1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "stateFormat":"2"} current state version: 3
   [junit4]   2> 1996529 INFO  (OverseerStateUpdate-96375247254716420-127.0.0.1:52303_hd_om-n_0000000000) [n:127.0.0.1:52303_hd_om    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 1996601 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_4C17D787E792729C-001/shard-1-001/cores/collection1
   [junit4]   2> 1996602 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_4C17D787E792729C-001/shard-1-001
   [junit4]   2> 1996602 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.e.j.s.Server jetty-9.3.8.v20160314
   [junit4]   2> 1996604 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@736f22fe{/hd_om,null,AVAILABLE}
   [junit4]   2> 1996604 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.e.j.s.ServerConnector Started ServerConnector@3c7f5e50{HTTP/1.1,[http/1.1]}{127.0.0.1:34903}
   [junit4]   2> 1996604 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.e.j.s.Server Started @1998815ms
   [junit4]   2> 1996604 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_4C17D787E792729C-001/tempDir-001/jetty1, solrconfig=solrconfig.xml, hostContext=/hd_om, hostPort=55743, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_4C17D787E792729C-001/shard-1-001/cores}
   [junit4]   2> 1996605 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@73d16e93
   [junit4]   2> 1996605 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_4C17D787E792729C-001/shard-1-001'
   [junit4]   2> 1996605 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1996605 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1996606 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1996606 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.

[...truncated too long message...]

s-6.x/solr/core/test-lib/apacheds-interceptors-schema-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-subtree-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-trigger-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-jdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-jdbm1-2.0.0-M2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-kerberos-codec-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-ldif-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-mavibot-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-protocol-kerberos-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-protocol-ldap-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-protocol-shared-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-xdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/api-all-1.0.0-M20.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/bcprov-jdk15-1.45.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/cglib-nodep-2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/commons-collections-3.2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/commons-math3-3.4.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/easymock-3.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/ehcache-core-2.4.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/hadoop-common-2.7.2-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/hadoop-hdfs-2.7.2-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/hadoop-minikdc-2.7.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/htrace-core-3.2.0-incubating.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jackson-annotations-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jackson-core-asl-1.9.13.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jackson-databind-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jackson-mapper-asl-1.9.13.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jersey-core-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jersey-server-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jetty-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jetty-util-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/mina-core-2.0.0-M5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/mockito-core-1.9.5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/netty-all-4.0.36.Final.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/objenesis-1.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/presto-parser-0.122.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/slice-0.10.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/analysis/icu/lucene-analyzers-icu-6.2.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/contrib/solr-analysis-extras/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/queryparser/classes/test:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/contrib/analysis-extras/lib/icu4j-56.1.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-launcher.jar:/x1/jenkins/.ant/lib/ivy-2.3.0.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-antlr.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-regexp.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-resolver.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-net.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jmf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-oro.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-xalan2.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-logging.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jsch.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit4.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-javamail.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jai.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jdepend.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bsf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-swing.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-netrexx.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bcel.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-testutil.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-log4j.jar:/x1/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.8/lib/tools.jar:/x1/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.3.4.jar com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/temp/junit4-J1-20160807_105930_686.events @/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/temp/junit4-J1-20160807_105930_686.suites -stdin
   [junit4] ERROR: JVM J1 ended with an exception: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1501)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:121)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:955)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:952)
   [junit4] 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4] 	at java.lang.Thread.run(Thread.java:745)

BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:771: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:707: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/common-build.xml:536: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/common-build.xml:1443: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/common-build.xml:984: At least one slave process threw an exception, first: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.

Total time: 237 minutes 24 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
ERROR: Failed to archive {README.txt=README.txt, heapdumps/java_pid21736.hprof=heapdumps/java_pid21736.hprof, solr/build/solr-core/test/temp/junit4-J2-20160807_105930_686.events=solr/build/solr-core/test/temp/junit4-J2-20160807_105930_686.events, solr/build/solr-core/test/temp/junit4-J1-20160807_105930_686.events=solr/build/solr-core/test/temp/junit4-J1-20160807_105930_686.events, solr/build/solr-core/test/temp/junit4-J0-20160807_105930_686.events=solr/build/solr-core/test/temp/junit4-J0-20160807_105930_686.events} due to internal error; falling back to full archiving
java.lang.NullPointerException
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any




[JENKINS] Lucene-Solr-NightlyTests-6.x - Build # 138 - Still unstable

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-6.x/138/

3 tests failed.
FAILED:  org.apache.solr.cloud.SaslZkACLProviderTest.testSaslZkACLProvider

Error Message:
java.net.BindException: Address already in use

Stack Trace:
java.lang.RuntimeException: java.net.BindException: Address already in use
	at org.apache.solr.cloud.SaslZkACLProviderTest$SaslZkTestServer.run(SaslZkACLProviderTest.java:208)
	at org.apache.solr.cloud.SaslZkACLProviderTest.setUp(SaslZkACLProviderTest.java:76)
	at sun.reflect.GeneratedMethodAccessor6.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:905)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.net.BindException: Address already in use
	at sun.nio.ch.Net.bind0(Native Method)
	at sun.nio.ch.Net.bind(Net.java:433)
	at sun.nio.ch.Net.bind(Net.java:425)
	at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223)
	at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
	at org.apache.mina.transport.socket.nio.NioSocketAcceptor.open(NioSocketAcceptor.java:252)
	at org.apache.mina.transport.socket.nio.NioSocketAcceptor.open(NioSocketAcceptor.java:49)
	at org.apache.mina.core.polling.AbstractPollingIoAcceptor.registerHandles(AbstractPollingIoAcceptor.java:525)
	at org.apache.mina.core.polling.AbstractPollingIoAcceptor.access$200(AbstractPollingIoAcceptor.java:67)
	at org.apache.mina.core.polling.AbstractPollingIoAcceptor$Acceptor.run(AbstractPollingIoAcceptor.java:409)
	at org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:65)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	... 1 more


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.SaslZkACLProviderTest

Error Message:
6 threads leaked from SUITE scope at org.apache.solr.cloud.SaslZkACLProviderTest:     1) Thread[id=15650, name=apacheds, state=WAITING, group=TGRP-SaslZkACLProviderTest]         at java.lang.Object.wait(Native Method)         at java.lang.Object.wait(Object.java:502)         at java.util.TimerThread.mainLoop(Timer.java:526)         at java.util.TimerThread.run(Timer.java:505)    2) Thread[id=15654, name=groupCache.data, state=TIMED_WAITING, group=TGRP-SaslZkACLProviderTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)    3) Thread[id=15655, name=pool-54-thread-1, state=TIMED_WAITING, group=TGRP-SaslZkACLProviderTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)    4) Thread[id=15652, name=ou=system.data, state=TIMED_WAITING, group=TGRP-SaslZkACLProviderTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)    5) Thread[id=15653, name=kdcReplayCache.data, state=TIMED_WAITING, group=TGRP-SaslZkACLProviderTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)    6) Thread[id=15651, name=changePwdReplayCache.data, state=TIMED_WAITING, group=TGRP-SaslZkACLProviderTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 6 threads leaked from SUITE scope at org.apache.solr.cloud.SaslZkACLProviderTest: 
   1) Thread[id=15650, name=apacheds, state=WAITING, group=TGRP-SaslZkACLProviderTest]
        at java.lang.Object.wait(Native Method)
        at java.lang.Object.wait(Object.java:502)
        at java.util.TimerThread.mainLoop(Timer.java:526)
        at java.util.TimerThread.run(Timer.java:505)
   2) Thread[id=15654, name=groupCache.data, state=TIMED_WAITING, group=TGRP-SaslZkACLProviderTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
   3) Thread[id=15655, name=pool-54-thread-1, state=TIMED_WAITING, group=TGRP-SaslZkACLProviderTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
   4) Thread[id=15652, name=ou=system.data, state=TIMED_WAITING, group=TGRP-SaslZkACLProviderTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
   5) Thread[id=15653, name=kdcReplayCache.data, state=TIMED_WAITING, group=TGRP-SaslZkACLProviderTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
   6) Thread[id=15651, name=changePwdReplayCache.data, state=TIMED_WAITING, group=TGRP-SaslZkACLProviderTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
	at __randomizedtesting.SeedInfo.seed([C8B0839AB9FE4F20]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.SaslZkACLProviderTest

Error Message:
There are still zombie threads that couldn't be terminated:    1) Thread[id=15650, name=apacheds, state=WAITING, group=TGRP-SaslZkACLProviderTest]         at java.lang.Object.wait(Native Method)         at java.lang.Object.wait(Object.java:502)         at java.util.TimerThread.mainLoop(Timer.java:526)         at java.util.TimerThread.run(Timer.java:505)    2) Thread[id=15654, name=groupCache.data, state=TIMED_WAITING, group=TGRP-SaslZkACLProviderTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)    3) Thread[id=15655, name=pool-54-thread-1, state=TIMED_WAITING, group=TGRP-SaslZkACLProviderTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)    4) Thread[id=15652, name=ou=system.data, state=TIMED_WAITING, group=TGRP-SaslZkACLProviderTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)    5) Thread[id=15653, name=kdcReplayCache.data, state=TIMED_WAITING, group=TGRP-SaslZkACLProviderTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)    6) Thread[id=15651, name=changePwdReplayCache.data, state=TIMED_WAITING, group=TGRP-SaslZkACLProviderTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   1) Thread[id=15650, name=apacheds, state=WAITING, group=TGRP-SaslZkACLProviderTest]
        at java.lang.Object.wait(Native Method)
        at java.lang.Object.wait(Object.java:502)
        at java.util.TimerThread.mainLoop(Timer.java:526)
        at java.util.TimerThread.run(Timer.java:505)
   2) Thread[id=15654, name=groupCache.data, state=TIMED_WAITING, group=TGRP-SaslZkACLProviderTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
   3) Thread[id=15655, name=pool-54-thread-1, state=TIMED_WAITING, group=TGRP-SaslZkACLProviderTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
   4) Thread[id=15652, name=ou=system.data, state=TIMED_WAITING, group=TGRP-SaslZkACLProviderTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
   5) Thread[id=15653, name=kdcReplayCache.data, state=TIMED_WAITING, group=TGRP-SaslZkACLProviderTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
   6) Thread[id=15651, name=changePwdReplayCache.data, state=TIMED_WAITING, group=TGRP-SaslZkACLProviderTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
	at __randomizedtesting.SeedInfo.seed([C8B0839AB9FE4F20]:0)




Build Log:
[...truncated 11465 lines...]
   [junit4] Suite: org.apache.solr.cloud.SaslZkACLProviderTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.SaslZkACLProviderTest_C8B0839AB9FE4F20-001/init-core-data-001
   [junit4]   2> 966786 INFO  (SUITE-SaslZkACLProviderTest-seed#[C8B0839AB9FE4F20]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
   [junit4]   2> 966793 INFO  (TEST-SaslZkACLProviderTest.testSaslZkACLProvider-seed#[C8B0839AB9FE4F20]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testSaslZkACLProvider
   [junit4]   2> 966793 INFO  (TEST-SaslZkACLProviderTest.testSaslZkACLProvider-seed#[C8B0839AB9FE4F20]) [    ] o.a.s.c.SaslZkACLProviderTest ####SETUP_START testSaslZkACLProvider
   [junit4]   2> 966793 INFO  (TEST-SaslZkACLProviderTest.testSaslZkACLProvider-seed#[C8B0839AB9FE4F20]) [    ] o.a.s.c.SaslZkACLProviderTest ZooKeeper dataDir:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.SaslZkACLProviderTest_C8B0839AB9FE4F20-001/tempDir-002/zookeeper/server1/data
   [junit4]   2> 971514 WARN  (TEST-SaslZkACLProviderTest.testSaslZkACLProvider-seed#[C8B0839AB9FE4F20]) [    ] o.a.d.s.c.DefaultDirectoryService You didn't change the admin password of directory service instance 'DefaultKrbServer'.  Please update the admin password as soon as possible to prevent a possible security breach.
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=SaslZkACLProviderTest -Dtests.method=testSaslZkACLProvider -Dtests.seed=C8B0839AB9FE4F20 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=no-NO -Dtests.timezone=Africa/Porto-Novo -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR   5.21s J2 | SaslZkACLProviderTest.testSaslZkACLProvider <<<
   [junit4]    > Throwable #1: java.lang.RuntimeException: java.net.BindException: Address already in use
   [junit4]    > 	at org.apache.solr.cloud.SaslZkACLProviderTest$SaslZkTestServer.run(SaslZkACLProviderTest.java:208)
   [junit4]    > 	at org.apache.solr.cloud.SaslZkACLProviderTest.setUp(SaslZkACLProviderTest.java:76)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]    > Caused by: java.net.BindException: Address already in use
   [junit4]    > 	at sun.nio.ch.Net.bind0(Native Method)
   [junit4]    > 	at sun.nio.ch.Net.bind(Net.java:433)
   [junit4]    > 	at sun.nio.ch.Net.bind(Net.java:425)
   [junit4]    > 	at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223)
   [junit4]    > 	at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
   [junit4]    > 	at org.apache.mina.transport.socket.nio.NioSocketAcceptor.open(NioSocketAcceptor.java:252)
   [junit4]    > 	at org.apache.mina.transport.socket.nio.NioSocketAcceptor.open(NioSocketAcceptor.java:49)
   [junit4]    > 	at org.apache.mina.core.polling.AbstractPollingIoAcceptor.registerHandles(AbstractPollingIoAcceptor.java:525)
   [junit4]    > 	at org.apache.mina.core.polling.AbstractPollingIoAcceptor.access$200(AbstractPollingIoAcceptor.java:67)
   [junit4]    > 	at org.apache.mina.core.polling.AbstractPollingIoAcceptor$Acceptor.run(AbstractPollingIoAcceptor.java:409)
   [junit4]    > 	at org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:65)
   [junit4]    > 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
   [junit4]    > 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]    > 	... 1 moreThrowable #2: java.lang.NullPointerException
   [junit4]    > 	at org.apache.solr.cloud.ZkTestServer$ZKServerMain.shutdown(ZkTestServer.java:334)
   [junit4]    > 	at org.apache.solr.cloud.ZkTestServer.shutdown(ZkTestServer.java:526)
   [junit4]    > 	at org.apache.solr.cloud.SaslZkACLProviderTest$SaslZkTestServer.shutdown(SaslZkACLProviderTest.java:215)
   [junit4]    > 	at org.apache.solr.cloud.SaslZkACLProviderTest.tearDown(SaslZkACLProviderTest.java:112)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 971999 INFO  (SUITE-SaslZkACLProviderTest-seed#[C8B0839AB9FE4F20]-worker) [    ] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.SaslZkACLProviderTest_C8B0839AB9FE4F20-001
   [junit4]   2> Aug 06, 2016 11:11:27 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 6 leaked thread(s).
   [junit4]   2> Aug 06, 2016 11:11:47 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> SEVERE: 6 threads leaked from SUITE scope at org.apache.solr.cloud.SaslZkACLProviderTest: 
   [junit4]   2>    1) Thread[id=15650, name=apacheds, state=WAITING, group=TGRP-SaslZkACLProviderTest]
   [junit4]   2>         at java.lang.Object.wait(Native Method)
   [junit4]   2>         at java.lang.Object.wait(Object.java:502)
   [junit4]   2>         at java.util.TimerThread.mainLoop(Timer.java:526)
   [junit4]   2>         at java.util.TimerThread.run(Timer.java:505)
   [junit4]   2>    2) Thread[id=15654, name=groupCache.data, state=TIMED_WAITING, group=TGRP-SaslZkACLProviderTest]
   [junit4]   2>         at sun.misc.Unsafe.park(Native Method)
   [junit4]   2>         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]   2>         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
   [junit4]   2>         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
   [junit4]   2>         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]   2>         at java.lang.Thread.run(Thread.java:745)
   [junit4]   2>    3) Thread[id=15655, name=pool-54-thread-1, state=TIMED_WAITING, group=TGRP-SaslZkACLProviderTest]
   [junit4]   2>         at sun.misc.Unsafe.park(Native Method)
   [junit4]   2>         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]   2>         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
   [junit4]   2>         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
   [junit4]   2>         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]   2>         at java.lang.Thread.run(Thread.java:745)
   [junit4]   2>    4) Thread[id=15652, name=ou=system.data, state=TIMED_WAITING, group=TGRP-SaslZkACLProviderTest]
   [junit4]   2>         at sun.misc.Unsafe.park(Native Method)
   [junit4]   2>         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]   2>         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
   [junit4]   2>         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
   [junit4]   2>         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]   2>         at java.lang.Thread.run(Thread.java:745)
   [junit4]   2>    5) Thread[id=15653, name=kdcReplayCache.data, state=TIMED_WAITING, group=TGRP-SaslZkACLProviderTest]
   [junit4]   2>         at sun.misc.Unsafe.park(Native Method)
   [junit4]   2>         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]   2>         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
   [junit4]   2>         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
   [junit4]   2>         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]   2>         at java.lang.Thread.run(Thread.java:745)
   [junit4]   2>    6) Thread[id=15651, name=changePwdReplayCache.data, state=TIMED_WAITING, group=TGRP-SaslZkACLProviderTest]
   [junit4]   2>         at sun.misc.Unsafe.park(Native Method)
   [junit4]   2>         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]   2>         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
   [junit4]   2>         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
   [junit4]   2>         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]   2>         at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> Aug 06, 2016 11:11:47 AM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
   [junit4]   2> INFO: Starting to interrupt leaked threads:
   [junit4]   2>    1) Thread[id=15650, name=apacheds, state=WAITING, group=TGRP-SaslZkACLProviderTest]
   [junit4]   2>    2) Thread[id=15654, name=groupCache.data, state=TIMED_WAITING, group=TGRP-SaslZkACLProviderTest]
   [junit4]   2>    3) Thread[id=15655, name=pool-54-thread-1, state=TIMED_WAITING, group=TGRP-SaslZkACLProviderTest]
   [junit4]   2>    4) Thread[id=15652, name=ou=system.data, state=TIMED_WAITING, group=TGRP-SaslZkACLProviderTest]
   [junit4]   2>    5) Thread[id=15653, name=kdcReplayCache.data, state=TIMED_WAITING, group=TGRP-SaslZkACLProviderTest]
   [junit4]   2>    6) Thread[id=15651, name=changePwdReplayCache.data, state=TIMED_WAITING, group=TGRP-SaslZkACLProviderTest]
   [junit4]   2> Aug 06, 2016 11:11:50 AM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
   [junit4]   2> SEVERE: There are still zombie threads that couldn't be terminated:
   [junit4]   2>    1) Thread[id=15650, name=apacheds, state=WAITING, group=TGRP-SaslZkACLProviderTest]
   [junit4]   2>         at java.lang.Object.wait(Native Method)
   [junit4]   2>         at java.lang.Object.wait(Object.java:502)
   [junit4]   2>         at java.util.TimerThread.mainLoop(Timer.java:526)
   [junit4]   2>         at java.util.TimerThread.run(Timer.java:505)
   [junit4]   2>    2) Thread[id=15654, name=groupCache.data, state=TIMED_WAITING, group=TGRP-SaslZkACLProviderTest]
   [junit4]   2>         at sun.misc.Unsafe.park(Native Method)
   [junit4]   2>         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]   2>         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
   [junit4]   2>         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
   [junit4]   2>         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]   2>         at java.lang.Thread.run(Thread.java:745)
   [junit4]   2>    3) Thread[id=15655, name=pool-54-thread-1, state=TIMED_WAITING, group=TGRP-SaslZkACLProviderTest]
   [junit4]   2>         at sun.misc.Unsafe.park(Native Method)
   [junit4]   2>         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]   2>         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
   [junit4]   2>         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
   [junit4]   2>         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]   2>         at java.lang.Thread.run(Thread.java:745)
   [junit4]   2>    4) Thread[id=15652, name=ou=system.data, state=TIMED_WAITING, group=TGRP-SaslZkACLProviderTest]
   [junit4]   2>         at sun.misc.Unsafe.park(Native Method)
   [junit4]   2>         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]   2>         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
   [junit4]   2>         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
   [junit4]   2>         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]   2>         at java.lang.Thread.run(Thread.java:745)
   [junit4]   2>    5) Thread[id=15653, name=kdcReplayCache.data, state=TIMED_WAITING, group=TGRP-SaslZkACLProviderTest]
   [junit4]   2>         at sun.misc.Unsafe.park(Native Method)
   [junit4]   2>         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]   2>         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
   [junit4]   2>         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
   [junit4]   2>         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]   2>         at java.lang.Thread.run(Thread.java:745)
   [junit4]   2>    6) Thread[id=15651, name=changePwdReplayCache.data, state=TIMED_WAITING, group=TGRP-SaslZkACLProviderTest]
   [junit4]   2>         at sun.misc.Unsafe.park(Native Method)
   [junit4]   2>         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]   2>         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
   [junit4]   2>         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
   [junit4]   2>         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]   2>         at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> NOTE: test params are: codec=DummyCompressingStoredFields(storedFieldsFormat=CompressingStoredFieldsFormat(compressionMode=DUMMY, chunkSize=6, maxDocsPerChunk=5, blockSize=700), termVectorsFormat=CompressingTermVectorsFormat(compressionMode=DUMMY, chunkSize=6, blockSize=700)), sim=RandomSimilarity(queryNorm=true,coord=yes): {}, locale=no-NO, timezone=Africa/Porto-Novo
   [junit4]   2> NOTE: Linux 3.13.0-85-generic amd64/Oracle Corporation 1.8.0_74 (64-bit)/cpus=4,threads=7,free=147174304,total=531628032
   [junit4]   2> NOTE: All tests run in this JVM: [LukeRequestHandlerTest, NoCacheHeaderTest, TestFilteredDocIdSet, RestartWhileUpdatingTest, TestAuthorizationFramework, TestBackupRepositoryFactory, TestSolrFieldCacheMBean, CircularListTest, CollectionStateFormat2Test, HardAutoCommitTest, StressHdfsTest, XmlUpdateRequestHandlerTest, TestCustomDocTransformer, SolrTestCaseJ4Test, TestLegacyFieldCache, BitVectorTest, TestPHPSerializedResponseWriter, StatelessScriptUpdateProcessorFactoryTest, TestRawResponseWriter, TestSubQueryTransformerCrossCore, TestSolrIndexConfig, TestTestInjection, SolrCoreCheckLockOnStartupTest, TestHdfsCloudBackupRestore, CurrencyFieldXmlFileTest, CleanupOldIndexTest, TestDFISimilarityFactory, AutoCommitTest, TestInitQParser, TestSortByMinMaxFunction, TestRecovery, TolerantUpdateProcessorTest, TestRecoveryHdfs, TestMiniSolrCloudClusterBase, TestDistributedMissingSort, DeleteShardTest, TestExtendedDismaxParser, TestStressRecovery, TestOnReconnectListenerSupport, TestNoOpRegenerator, TestManagedResourceStorage, UnloadDistributedZkTest, SuggestComponentTest, TestSort, TestManagedSynonymFilterFactory, FacetPivotSmallTest, TestCollationFieldDocValues, TestJmxIntegration, SliceStateTest, TestConfigSetProperties, SuggesterTest, TestCoreDiscovery, SharedFSAutoReplicaFailoverUtilsTest, AnalyticsMergeStrategyTest, DistributedQueryComponentCustomSortTest, TestJettySolrRunner, TestZkChroot, LeaderFailoverAfterPartitionTest, TestSimpleQParserPlugin, TestSchemaNameResource, DocValuesMultiTest, TestSolrConfigHandler, OverseerRolesTest, MergeStrategyTest, CheckHdfsIndexTest, TestReqParamsAPI, SaslZkACLProviderTest]
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=SaslZkACLProviderTest -Dtests.seed=C8B0839AB9FE4F20 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=no-NO -Dtests.timezone=Africa/Porto-Novo -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR   0.00s J2 | SaslZkACLProviderTest (suite) <<<
   [junit4]    > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 6 threads leaked from SUITE scope at org.apache.solr.cloud.SaslZkACLProviderTest: 
   [junit4]    >    1) Thread[id=15650, name=apacheds, state=WAITING, group=TGRP-SaslZkACLProviderTest]
   [junit4]    >         at java.lang.Object.wait(Native Method)
   [junit4]    >         at java.lang.Object.wait(Object.java:502)
   [junit4]    >         at java.util.TimerThread.mainLoop(Timer.java:526)
   [junit4]    >         at java.util.TimerThread.run(Timer.java:505)
   [junit4]    >    2) Thread[id=15654, name=groupCache.data, state=TIMED_WAITING, group=TGRP-SaslZkACLProviderTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
   [junit4]    >         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
   [junit4]    >         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >    3) Thread[id=15655, name=pool-54-thread-1, state=TIMED_WAITING, group=TGRP-SaslZkACLProviderTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
   [junit4]    >         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >    4) Thread[id=15652, name=ou=system.data, state=TIMED_WAITING, group=TGRP-SaslZkACLProviderTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
   [junit4]    >         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
   [junit4]    >         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >    5) Thread[id=15653, name=kdcReplayCache.data, state=TIMED_WAITING, group=TGRP-SaslZkACLProviderTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
   [junit4]    >         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
   [junit4]    >         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >    6) Thread[id=15651, name=changePwdReplayCache.data, state=TIMED_WAITING, group=TGRP-SaslZkACLProviderTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
   [junit4]    >         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
   [junit4]    >         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([C8B0839AB9FE4F20]:0)Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   [junit4]    >    1) Thread[id=15650, name=apacheds, state=WAITING, group=TGRP-SaslZkACLProviderTest]
   [junit4]    >         at java.lang.Object.wait(Native Method)
   [junit4]    >         at java.lang.Object.wait(Object.java:502)
   [junit4]    >         at java.util.TimerThread.mainLoop(Timer.java:526)
   [junit4]    >         at java.util.TimerThread.run(Timer.java:505)
   [junit4]    >    2) Thread[id=15654, name=groupCache.data, state=TIMED_WAITING, group=TGRP-SaslZkACLProviderTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
   [junit4]    >         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
   [junit4]    >         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >    3) Thread[id=15655, name=pool-54-thread-1, state=TIMED_WAITING, group=TGRP-SaslZkACLProviderTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
   [junit4]    >         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >    4) Thread[id=15652, name=ou=system.data, state=TIMED_WAITING, group=TGRP-SaslZkACLProviderTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
   [junit4]    >         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
   [junit4]    >         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >    5) Thread[id=15653, name=kdcReplayCache.data, state=TIMED_WAITING, group=TGRP-SaslZkACLProviderTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
   [junit4]    >         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
   [junit4]    >         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >    6) Thread[id=15651, name=changePwdReplayCache.data, state=TIMED_WAITING, group=TGRP-SaslZkACLProviderTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
   [junit4]    >         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
   [junit4]    >         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([C8B0839AB9FE4F20]:0)
   [junit4] Completed [125/626 (1!)] on J2 in 28.53s, 1 test, 3 errors <<< FAILURES!

[...truncated 61614 lines...]



[JENKINS] Lucene-Solr-NightlyTests-6.x - Build # 137 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-6.x/137/

1 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.handler.TestReplicationHandler

Error Message:
ObjectTracker found 1 object(s) that were not released!!! [NRTCachingDirectory]

Stack Trace:
java.lang.AssertionError: ObjectTracker found 1 object(s) that were not released!!! [NRTCachingDirectory]
	at __randomizedtesting.SeedInfo.seed([CC4C799E8C562B1C]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.junit.Assert.assertNull(Assert.java:551)
	at org.apache.solr.SolrTestCaseJ4.teardownTestCases(SolrTestCaseJ4.java:258)
	at sun.reflect.GeneratedMethodAccessor22.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:834)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 11732 lines...]
   [junit4] Suite: org.apache.solr.handler.TestReplicationHandler
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/init-core-data-001
   [junit4]   2> 809140 INFO  (SUITE-TestReplicationHandler-seed#[CC4C799E8C562B1C]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=None)
   [junit4] IGNOR/A 0.00s J0 | TestReplicationHandler.doTestIndexFetchOnMasterRestart
   [junit4]    > Assumption #1: 'awaitsfix' test group is disabled (@AwaitsFix(bugUrl=https://issues.apache.org/jira/browse/SOLR-9036))
   [junit4]   2> 809142 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.SolrTestCaseJ4 ###Starting doTestReplicateAfterStartup
   [junit4]   2> 809142 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-001/collection1
   [junit4]   2> 809144 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.e.j.s.Server jetty-9.3.8.v20160314
   [junit4]   2> 809146 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@4d34093{/solr,null,AVAILABLE}
   [junit4]   2> 809147 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.e.j.s.ServerConnector Started ServerConnector@3e4334b6{HTTP/1.1,[http/1.1]}{127.0.0.1:56861}
   [junit4]   2> 809147 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.e.j.s.Server Started @811759ms
   [junit4]   2> 809147 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-001/collection1/data, hostContext=/solr, hostPort=56861}
   [junit4]   2> 809147 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@73d16e93
   [junit4]   2> 809147 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-001'
   [junit4]   2> 809147 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 809147 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 809148 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-001/solr.xml
   [junit4]   2> 809151 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-001/.
   [junit4]   2> 809151 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.c.CoreContainer New CoreContainer 1942010471
   [junit4]   2> 809151 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-001]
   [junit4]   2> 809151 WARN  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-001/lib
   [junit4]   2> 809152 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,connectionsEvictorSleepDelay : 5000,maxConnectionIdleTime : 40000,
   [junit4]   2> 809153 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 809153 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 809153 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 809153 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 809153 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 809153 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-001/.
   [junit4]   2> 809154 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {name=collection1, config=solrconfig.xml, loadOnStartup=true, schema=schema.xml, configSetProperties=configsetprops.json, transient=false, dataDir=data/}
   [junit4]   2> 809154 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-001/./collection1
   [junit4]   2> 809154 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 809158 INFO  (coreLoadExecutor-1559-thread-1) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-001/collection1'
   [junit4]   2> 809158 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0
   [junit4]   2> 809159 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 809159 INFO  (coreLoadExecutor-1559-thread-1) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 809159 INFO  (coreLoadExecutor-1559-thread-1) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 809163 INFO  (coreLoadExecutor-1559-thread-1) [    ] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 809165 WARN  (coreLoadExecutor-1559-thread-1) [    ] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
   [junit4]   2> 809165 INFO  (coreLoadExecutor-1559-thread-1) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.2.0
   [junit4]   2> 809170 INFO  (coreLoadExecutor-1559-thread-1) [    ] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 809171 INFO  (coreLoadExecutor-1559-thread-1) [    ] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 809173 INFO  (coreLoadExecutor-1559-thread-1) [    ] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 809174 INFO  (coreLoadExecutor-1559-thread-1) [    ] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-001/collection1'
   [junit4]   2> 809174 INFO  (coreLoadExecutor-1559-thread-1) [    ] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-001/./collection1
   [junit4]   2> 809174 INFO  (coreLoadExecutor-1559-thread-1) [    x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 809175 INFO  (coreLoadExecutor-1559-thread-1) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-001/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-001/./collection1/data/]
   [junit4]   2> 809175 INFO  (coreLoadExecutor-1559-thread-1) [    x:collection1] o.a.s.c.SolrCore JMX monitoring not detected for core: collection1
   [junit4]   2> 809175 INFO  (coreLoadExecutor-1559-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-001/./collection1/data/snapshot_metadata
   [junit4]   2> 809175 INFO  (coreLoadExecutor-1559-thread-1) [    x:collection1] o.a.s.c.s.SolrSnapshotMetaDataManager Loading from snapshot metadata file...
   [junit4]   2> 809175 INFO  (coreLoadExecutor-1559-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-001/./collection1/data
   [junit4]   2> 809175 INFO  (coreLoadExecutor-1559-thread-1) [    x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-001/./collection1/data/index/
   [junit4]   2> 809175 WARN  (coreLoadExecutor-1559-thread-1) [    x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-001/./collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 809175 INFO  (coreLoadExecutor-1559-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-001/./collection1/data/index
   [junit4]   2> 809175 INFO  (coreLoadExecutor-1559-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=15, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
   [junit4]   2> 809176 INFO  (coreLoadExecutor-1559-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(NRTCachingDirectory(RAMDirectory@253a50c2 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@6f5a06ec; maxCacheMB=0.87109375 maxMergeSizeMB=0.3525390625)),segFN=segments_1,generation=1}
   [junit4]   2> 809176 INFO  (coreLoadExecutor-1559-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 809176 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [    x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-001/./collection1/data/
   [junit4]   2> 809177 INFO  (coreLoadExecutor-1559-thread-1) [    x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 809178 INFO  (coreLoadExecutor-1559-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 809179 INFO  (coreLoadExecutor-1559-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 809179 INFO  (coreLoadExecutor-1559-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 809180 INFO  (coreLoadExecutor-1559-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 809180 INFO  (coreLoadExecutor-1559-thread-1) [    x:collection1] o.a.s.c.PluginBag requestHandler : 'lazy' created with startup=lazy 
   [junit4]   2> 809180 INFO  (coreLoadExecutor-1559-thread-1) [    x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/mbeans,standard,lazy,/update/json/docs,/admin/luke,/export,/get,/admin/properties,/update/json,/admin/threads,/update/csv,/sql,/graph,/admin/segments,/admin/system,/replication,defaults,/config,/stream,/schema,/admin/plugins,/admin/logging,/admin/ping,/update,/admin/file,/terms
   [junit4]   2> 809180 INFO  (coreLoadExecutor-1559-thread-1) [    x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 809181 INFO  (coreLoadExecutor-1559-thread-1) [    x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 809181 INFO  (coreLoadExecutor-1559-thread-1) [    x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 809181 INFO  (coreLoadExecutor-1559-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=47, maxMergeAtOnceExplicit=47, maxMergedSegmentMB=24.6630859375, floorSegmentMB=1.19140625, forceMergeDeletesPctAllowed=23.02471689484807, segmentsPerTier=39.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.4020035669670132
   [junit4]   2> 809181 INFO  (coreLoadExecutor-1559-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(NRTCachingDirectory(RAMDirectory@253a50c2 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@6f5a06ec; maxCacheMB=0.87109375 maxMergeSizeMB=0.3525390625)),segFN=segments_1,generation=1}
   [junit4]   2> 809181 INFO  (coreLoadExecutor-1559-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 809181 INFO  (coreLoadExecutor-1559-thread-1) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@42e0fd9f[collection1] main]
   [junit4]   2> 809181 INFO  (coreLoadExecutor-1559-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-001/collection1/conf
   [junit4]   2> 809181 INFO  (coreLoadExecutor-1559-thread-1) [    x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {storageDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-001/collection1/conf}
   [junit4]   2> 809181 INFO  (coreLoadExecutor-1559-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using file:dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-001/collection1/conf
   [junit4]   2> 809181 INFO  (coreLoadExecutor-1559-thread-1) [    x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 809181 INFO  (coreLoadExecutor-1559-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler Replication enabled for following config files: schema.xml,xslt/dummy.xsl
   [junit4]   2> 809181 INFO  (coreLoadExecutor-1559-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 809182 INFO  (searcherExecutor-1560-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@42e0fd9f[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 809182 INFO  (coreLoadExecutor-1559-thread-1) [    x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 809660 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-002/collection1
   [junit4]   2> 809662 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.e.j.s.Server jetty-9.3.8.v20160314
   [junit4]   2> 809663 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5898ea8b{/solr,null,AVAILABLE}
   [junit4]   2> 809663 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.e.j.s.ServerConnector Started ServerConnector@deef306{HTTP/1.1,[http/1.1]}{127.0.0.1:38139}
   [junit4]   2> 809663 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.e.j.s.Server Started @812275ms
   [junit4]   2> 809663 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-002/collection1/data, hostContext=/solr, hostPort=38139}
   [junit4]   2> 809664 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@73d16e93
   [junit4]   2> 809664 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-002'
   [junit4]   2> 809664 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 809664 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 809664 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-002/solr.xml
   [junit4]   2> 809667 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-002/.
   [junit4]   2> 809667 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.c.CoreContainer New CoreContainer 1817000418
   [junit4]   2> 809667 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-002]
   [junit4]   2> 809668 WARN  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-002/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-002/lib
   [junit4]   2> 809668 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,connectionsEvictorSleepDelay : 5000,maxConnectionIdleTime : 40000,
   [junit4]   2> 809669 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 809669 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 809669 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 809669 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 809669 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 809669 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-002/.
   [junit4]   2> 809670 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {name=collection1, config=solrconfig.xml, loadOnStartup=true, schema=schema.xml, configSetProperties=configsetprops.json, transient=false, dataDir=data/}
   [junit4]   2> 809670 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-002/./collection1
   [junit4]   2> 809670 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 809670 INFO  (coreLoadExecutor-1569-thread-1) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-002/collection1'
   [junit4]   2> 809671 INFO  (coreLoadExecutor-1569-thread-1) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 809671 INFO  (coreLoadExecutor-1569-thread-1) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 809671 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0
   [junit4]   2> 809671 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 809674 INFO  (coreLoadExecutor-1569-thread-1) [    ] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 809682 WARN  (coreLoadExecutor-1569-thread-1) [    ] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
   [junit4]   2> 809683 INFO  (coreLoadExecutor-1569-thread-1) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.2.0
   [junit4]   2> 809687 INFO  (coreLoadExecutor-1569-thread-1) [    ] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 809689 INFO  (coreLoadExecutor-1569-thread-1) [    ] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 809690 INFO  (coreLoadExecutor-1569-thread-1) [    ] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 809691 INFO  (coreLoadExecutor-1569-thread-1) [    ] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-002/collection1'
   [junit4]   2> 809691 INFO  (coreLoadExecutor-1569-thread-1) [    ] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-002/./collection1
   [junit4]   2> 809691 INFO  (coreLoadExecutor-1569-thread-1) [    x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 809692 INFO  (coreLoadExecutor-1569-thread-1) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-002/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-002/./collection1/data/]
   [junit4]   2> 809692 INFO  (coreLoadExecutor-1569-thread-1) [    x:collection1] o.a.s.c.SolrCore JMX monitoring not detected for core: collection1
   [junit4]   2> 809692 INFO  (coreLoadExecutor-1569-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-002/./collection1/data/snapshot_metadata
   [junit4]   2> 809692 INFO  (coreLoadExecutor-1569-thread-1) [    x:collection1] o.a.s.c.s.SolrSnapshotMetaDataManager Loading from snapshot metadata file...
   [junit4]   2> 809692 INFO  (coreLoadExecutor-1569-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-002/./collection1/data
   [junit4]   2> 809692 INFO  (coreLoadExecutor-1569-thread-1) [    x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-002/./collection1/data/index/
   [junit4]   2> 809692 WARN  (coreLoadExecutor-1569-thread-1) [    x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-002/./collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 809692 INFO  (coreLoadExecutor-1569-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-002/./collection1/data/index
   [junit4]   2> 809692 INFO  (coreLoadExecutor-1569-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=15, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
   [junit4]   2> 809693 INFO  (coreLoadExecutor-1569-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(NRTCachingDirectory(RAMDirectory@77545013 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@7bf82d3d; maxCacheMB=0.87109375 maxMergeSizeMB=0.3525390625)),segFN=segments_1,generation=1}
   [junit4]   2> 809693 INFO  (coreLoadExecutor-1569-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 809693 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [    x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-002/./collection1/data/
   [junit4]   2> 809694 INFO  (coreLoadExecutor-1569-thread-1) [    x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 809695 INFO  (coreLoadExecutor-1569-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 809695 INFO  (coreLoadExecutor-1569-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 809696 INFO  (coreLoadExecutor-1569-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 809697 INFO  (coreLoadExecutor-1569-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 809697 INFO  (coreLoadExecutor-1569-thread-1) [    x:collection1] o.a.s.c.PluginBag requestHandler : 'lazy' created with startup=lazy 
   [junit4]   2> 809697 INFO  (coreLoadExecutor-1569-thread-1) [    x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/mbeans,standard,lazy,/update/json/docs,/admin/luke,/export,/get,/admin/properties,/update/json,/admin/threads,/update/csv,/sql,/graph,/admin/segments,/admin/system,/replication,defaults,/config,/stream,/schema,/admin/plugins,/admin/logging,/admin/ping,/update,/admin/file,/terms
   [junit4]   2> 809698 INFO  (coreLoadExecutor-1569-thread-1) [    x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 809698 INFO  (coreLoadExecutor-1569-thread-1) [    x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 809698 INFO  (coreLoadExecutor-1569-thread-1) [    x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 809698 INFO  (coreLoadExecutor-1569-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=47, maxMergeAtOnceExplicit=47, maxMergedSegmentMB=24.6630859375, floorSegmentMB=1.19140625, forceMergeDeletesPctAllowed=23.02471689484807, segmentsPerTier=39.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.4020035669670132
   [junit4]   2> 809698 INFO  (coreLoadExecutor-1569-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(NRTCachingDirectory(RAMDirectory@77545013 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@7bf82d3d; maxCacheMB=0.87109375 maxMergeSizeMB=0.3525390625)),segFN=segments_1,generation=1}
   [junit4]   2> 809698 INFO  (coreLoadExecutor-1569-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 809698 INFO  (coreLoadExecutor-1569-thread-1) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@7fbd81b2[collection1] main]
   [junit4]   2> 809698 INFO  (coreLoadExecutor-1569-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-002/collection1/conf
   [junit4]   2> 809698 INFO  (coreLoadExecutor-1569-thread-1) [    x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {storageDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-002/collection1/conf}
   [junit4]   2> 809698 INFO  (coreLoadExecutor-1569-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using file:dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-002/collection1/conf
   [junit4]   2> 809698 INFO  (coreLoadExecutor-1569-thread-1) [    x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 809700 INFO  (coreLoadExecutor-1569-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler Poll scheduled at an interval of 1000ms
   [junit4]   2> 809700 INFO  (searcherExecutor-1570-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@7fbd81b2[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 809700 INFO  (coreLoadExecutor-1569-thread-1) [    x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 809902 INFO  (qtp1677376140-50770) [    x:collection1] o.a.s.c.S.Request [collection1]  webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=0
   [junit4]   2> 810176 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.e.j.s.ServerConnector Stopped ServerConnector@deef306{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
   [junit4]   2> 810176 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1817000418
   [junit4]   2> 810177 INFO  (coreCloseExecutor-1575-thread-1) [    x:collection1] o.a.s.c.SolrCore [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@284f9677
   [junit4]   2> 810177 INFO  (coreCloseExecutor-1575-thread-1) [    x:collection1] o.a.s.u.DirectUpdateHandler2 closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
   [junit4]   2> 810177 INFO  (coreCloseExecutor-1575-thread-1) [    x:collection1] o.a.s.u.SolrCoreState Closing SolrCoreState
   [junit4]   2> 810177 INFO  (coreCloseExecutor-1575-thread-1) [    x:collection1] o.a.s.u.DefaultSolrCoreState SolrCoreState ref count has reached 0 - closing IndexWriter
   [junit4]   2> 810177 INFO  (coreCloseExecutor-1575-thread-1) [    x:collection1] o.a.s.u.DefaultSolrCoreState closing IndexWriter with IndexWriterCloser
   [junit4]   2> 810178 INFO  (coreCloseExecutor-1575-thread-1) [    x:collection1] o.a.s.c.SolrCore [collection1] Closing main searcher on request.
   [junit4]   2> 810178 INFO  (coreCloseExecutor-1575-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing MockDirectoryFactory - 3 directories currently being tracked
   [junit4]   2> 810178 INFO  (coreCloseExecutor-1575-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory looking to close /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-002/./collection1/data/snapshot_metadata [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-002/./collection1/data/snapshot_metadata;done=false>>]
   [junit4]   2> 810178 INFO  (coreCloseExecutor-1575-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory, CoreContainer#isShutdown=true
   [junit4]   2> 810178 INFO  (coreCloseExecutor-1575-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-002/./collection1/data/snapshot_metadata
   [junit4]   2> 810178 INFO  (coreCloseExecutor-1575-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory looking to close /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-002/./collection1/data [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-002/./collection1/data;done=false>>]
   [junit4]   2> 810178 INFO  (coreCloseExecutor-1575-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory, CoreContainer#isShutdown=true
   [junit4]   2> 810178 INFO  (coreCloseExecutor-1575-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-002/./collection1/data
   [junit4]   2> 810178 INFO  (coreCloseExecutor-1575-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory looking to close /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-002/./collection1/data/index [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-002/./collection1/data/index;done=false>>]
   [junit4]   2> 810178 INFO  (coreCloseExecutor-1575-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory, CoreContainer#isShutdown=true
   [junit4]   2> 810178 INFO  (coreCloseExecutor-1575-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-002/./collection1/data/index
   [junit4]   2> 810179 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@5898ea8b{/solr,null,UNAVAILABLE}
   [junit4]   2> 810181 INFO  (qtp1677376140-50773) [    x:collection1] o.a.s.u.DirectUpdateHandler2 [collection1] REMOVING ALL DOCUMENTS FROM INDEX
   [junit4]   2> 810181 INFO  (qtp1677376140-50773) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{deleteByQuery=*:*} 0 0
   [junit4]   2> 810184 INFO  (qtp1677376140-50773) [    x:collection1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 810184 INFO  (qtp1677376140-50773) [    x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(NRTCachingDirectory(RAMDirectory@253a50c2 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@6f5a06ec; maxCacheMB=0.87109375 maxMergeSizeMB=0.3525390625)),segFN=segments_1,generation=1}
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(NRTCachingDirectory(RAMDirectory@253a50c2 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@6f5a06ec; maxCacheMB=0.87109375 maxMergeSizeMB=0.3525390625)),segFN=segments_2,generation=2}
   [junit4]   2> 810184 INFO  (qtp1677376140-50773) [    x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 2
   [junit4]   2> 810184 INFO  (qtp1677376140-50773) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@445ca1a5[collection1] main]
   [junit4]   2> 810184 INFO  (qtp1677376140-50773) [    x:collection1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 810184 INFO  (searcherExecutor-1560-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@445ca1a5[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 810185 INFO  (qtp1677376140-50773) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2}{commit=} 0 1
   [junit4]   2> 810191 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.e.j.s.ServerConnector Stopped ServerConnector@3e4334b6{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
   [junit4]   2> 810191 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1942010471
   [junit4]   2> 810196 INFO  (coreCloseExecutor-1577-thread-1) [    x:collection1] o.a.s.c.SolrCore [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@4b506091
   [junit4]   2> 810196 INFO  (coreCloseExecutor-1577-thread-1) [    x:collection1] o.a.s.u.DirectUpdateHandler2 closing DirectUpdateHandler2{commits=1,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=1,cumulative_errors=0}
   [junit4]   2> 810197 INFO  (coreCloseExecutor-1577-thread-1) [    x:collection1] o.a.s.u.SolrCoreState Closing SolrCoreState
   [junit4]   2> 810197 INFO  (coreCloseExecutor-1577-thread-1) [    x:collection1] o.a.s.u.DefaultSolrCoreState SolrCoreState ref count has reached 0 - closing IndexWriter
   [junit4]   2> 810197 INFO  (coreCloseExecutor-1577-thread-1) [    x:collection1] o.a.s.u.DefaultSolrCoreState closing IndexWriter with IndexWriterCloser
   [junit4]   2> 810197 INFO  (coreCloseExecutor-1577-thread-1) [    x:collection1] o.a.s.c.SolrCore [collection1] Closing main searcher on request.
   [junit4]   2> 810197 INFO  (coreCloseExecutor-1577-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing MockDirectoryFactory - 3 directories currently being tracked
   [junit4]   2> 810197 INFO  (coreCloseExecutor-1577-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory looking to close /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-001/./collection1/data [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-001/./collection1/data;done=false>>]
   [junit4]   2> 810197 INFO  (coreCloseExecutor-1577-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory, CoreContainer#isShutdown=true
   [junit4]   2> 810197 INFO  (coreCloseExecutor-1577-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-001/./collection1/data
   [junit4]   2> 810197 INFO  (coreCloseExecutor-1577-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory looking to close /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-001/./collection1/data/index [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-001/./collection1/data/index;done=false>>]
   [junit4]   2> 810197 INFO  (coreCloseExecutor-1577-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory, CoreContainer#isShutdown=true
   [junit4]   2> 810197 INFO  (coreCloseExecutor-1577-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-001/./collection1/data/index
   [junit4]   2> 810197 INFO  (coreCloseExecutor-1577-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory looking to close /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-001/./collection1/data/snapshot_metadata [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-001/./collection1/data/snapshot_metadata;done=false>>]
   [junit4]   2> 810197 INFO  (coreCloseExecutor-1577-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory, CoreContainer#isShutdown=true
   [junit4]   2> 810197 INFO  (coreCloseExecutor-1577-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-001/./collection1/data/snapshot_metadata
   [junit4]   2> 810202 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@4d34093{/solr,null,UNAVAILABLE}
   [junit4]   2> 810203 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.e.j.s.Server jetty-9.3.8.v20160314
   [junit4]   2> 810204 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@20ad5f7f{/solr,null,AVAILABLE}
   [junit4]   2> 810204 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.e.j.s.ServerConnector Started ServerConnector@6c4d4310{HTTP/1.1,[http/1.1]}{127.0.0.1:33173}
   [junit4]   2> 810204 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.e.j.s.Server Started @812817ms
   [junit4]   2> 810205 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-001/collection1/data, hostContext=/solr, hostPort=33173}
   [junit4]   2> 810205 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@73d16e93
   [junit4]   2> 810205 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-001'
   [junit4]   2> 810205 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 810205 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 810205 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-001/solr.xml
   [junit4]   2> 810210 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-001/.
   [junit4]   2> 810210 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.c.CoreContainer New CoreContainer 650691908
   [junit4]   2> 810210 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-001]
   [junit4]   2> 810210 WARN  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-001/lib
   [junit4]   2> 810210 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,connectionsEvictorSleepDelay : 5000,maxConnectionIdleTime : 40000,
   [junit4]   2> 810221 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 810225 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 810225 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 810227 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 810227 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 810227 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-001/.
   [junit4]   2> 810228 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {name=collection1, config=solrconfig.xml, loadOnStartup=true, schema=schema.xml, configSetProperties=configsetprops.json, transient=false, dataDir=data/}
   [junit4]   2> 810228 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-001/./collection1
   [junit4]   2> 810228 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 810230 INFO  (coreLoadExecutor-1584-thread-1) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-001/collection1'
   [junit4]   2> 810237 INFO  (coreLoadExecutor-1584-thread-1) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 810237 INFO  (coreLoadExecutor-1584-thread-1) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 810231 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0
   [junit4]   2> 810244 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 810247 INFO  (coreLoadExecutor-1584-thread-1) [    ] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 810249 WARN  (coreLoadExecutor-1584-thread-1) [    ] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
   [junit4]   2> 810249 INFO  (coreLoadExecutor-1584-thread-1) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.2.0
   [junit4]   2> 810254 INFO  (coreLoadExecutor-1584-thread-1) [    ] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 810256 INFO  (coreLoadExecutor-1584-thread-1) [    ] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 810260 INFO  (coreLoadExecutor-1584-thread-1) [    ] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 810261 INFO  (coreLoadExecutor-1584-thread-1) [    ] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-001/collection1'
   [junit4]   2> 810261 INFO  (coreLoadExecutor-1584-thread-1) [    ] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-001/./collection1
   [junit4]   2> 810261 INFO  (coreLoadExecutor-1584-thread-1) [    x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 810261 INFO  (coreLoadExecutor-1584-thread-1) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-001/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-001/./collection1/data/]
   [junit4]   2> 810261 INFO  (coreLoadExecutor-1584-thread-1) [    x:collection1] o.a.s.c.SolrCore JMX monitoring not detected for core: collection1
   [junit4]   2> 810261 INFO  (coreLoadExecutor-1584-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-001/./collection1/data/snapshot_metadata
   [junit4]   2> 810261 INFO  (coreLoadExecutor-1584-thread-1) [    x:collection1] o.a.s.c.s.SolrSnapshotMetaDataManager Loading from snapshot metadata file...
   [junit4]   2> 810261 INFO  (coreLoadExecutor-1584-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-001/./collection1/data
   [junit4]   2> 810261 INFO  (coreLoadExecutor-1584-thread-1) [    x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-001/./collection1/data/index/
   [junit4]   2> 810261 WARN  (coreLoadExecutor-1584-thread-1) [    x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-001/./collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 810262 INFO  (coreLoadExecutor-1584-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-001/./collection1/data/index
   [junit4]   2> 810262 INFO  (coreLoadExecutor-1584-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=15, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
   [junit4]   2> 810262 INFO  (coreLoadExecutor-1584-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(NRTCachingDirectory(RAMDirectory@4f1b47fc lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@6dd00a54; maxCacheMB=0.87109375 maxMergeSizeMB=0.3525390625)),segFN=segments_1,generation=1}
   [junit4]   2> 810262 INFO  (coreLoadExecutor-1584-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 810262 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [    x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-001/./collection1/data/
   [junit4]   2> 810263 INFO  (coreLoadExecutor-1584-thread-1) [    x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 810264 INFO  (coreLoadExecutor-1584-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 810264 INFO  (coreLoadExecutor-1584-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 810265 INFO  (coreLoadExecutor-1584-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 810265 INFO  (coreLoadExecutor-1584-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 810266 INFO  (coreLoadExecutor-1584-thread-1) [    x:collection1] o.a.s.c.PluginBag requestHandler : 'lazy' created with startup=lazy 
   [junit4]   2> 810266 INFO  (coreLoadExecutor-1584-thread-1) [    x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/mbeans,standard,lazy,/update/json/docs,/admin/luke,/export,/get,/admin/properties,/update/json,/admin/threads,/update/csv,/sql,/graph,/admin/segments,/admin/system,/replication,defaults,/config,/stream,/schema,/admin/plugins,/admin/logging,/admin/ping,/update,/admin/file,/terms
   [junit4]   2> 810266 INFO  (coreLoadExecutor-1584-thread-1) [    x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 810266 INFO  (coreLoadExecutor-1584-thread-1) [    x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 810266 INFO  (coreLoadExecutor-1584-thread-1) [    x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 810266 INFO  (coreLoadExecutor-1584-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=47, maxMergeAtOnceExplicit=47, maxMergedSegmentMB=24.6630859375, floorSegmentMB=1.19140625, forceMergeDeletesPctAllowed=23.02471689484807, segmentsPerTier=39.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.4020035669670132
   [junit4]   2> 810266 INFO  (coreLoadExecutor-1584-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(NRTCachingDirectory(RAMDirectory@4f1b47fc lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@6dd00a54; maxCacheMB=0.87109375 maxMergeSizeMB=0.3525390625)),segFN=segments_1,generation=1}
   [junit4]   2> 810266 INFO  (coreLoadExecutor-1584-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 810267 INFO  (coreLoadExecutor-1584-thread-1) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@40414c6e[collection1] main]
   [junit4]   2> 810267 INFO  (coreLoadExecutor-1584-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-001/collection1/conf
   [junit4]   2> 810267 INFO  (coreLoadExecutor-1584-thread-1) [    x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {storageDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-001/collection1/conf}
   [junit4]   2> 810267 INFO  (coreLoadExecutor-1584-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using file:dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-001/collection1/conf
   [junit4]   2> 810267 INFO  (coreLoadExecutor-1584-thread-1) [    x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 810267 INFO  (coreLoadExecutor-1584-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler Replication enabled for following config files: schema.xml
   [junit4]   2> 810267 INFO  (coreLoadExecutor-1584-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 810267 INFO  (coreLoadExecutor-1584-thread-1) [    x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 810267 INFO  (searcherExecutor-1585-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@40414c6e[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 810750 INFO  (qtp990030861-50808) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[0]} 0 3
   [junit4]   2> 810751 INFO  (qtp990030861-50809) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[1]} 0 0
   [junit4]   2> 810753 INFO  (qtp990030861-50809) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[2]} 0 0
   [junit4]   2> 810754 INFO  (qtp990030861-50811) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[3]} 0 0
   [junit4]   2> 810756 INFO  (qtp990030861-50812) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[4]} 0 0
   [junit4]   2> 810757 INFO  (qtp990030861-50805) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[5]} 0 0
   [junit4]   2> 810759 INFO  (qtp990030861-50808) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[6]} 0 0
   [junit4]   2> 810760 INFO  (qtp990030861-50810) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[7]} 0 0
   [junit4]   2> 810762 INFO  (qtp990030861-50809) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[8]} 0 0
   [junit4]   2> 810763 INFO  (qtp990030861-50811) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[9]} 0 0
   [junit4]   2> 810764 INFO  (qtp990030861-50812) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[10]} 0 0
   [junit4]   2> 810766 INFO  (qtp990030861-50805) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[11]} 0 0
   [junit4]   2> 810767 INFO  (qtp990030861-50808) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[12]} 0 0
   [junit4]   2> 810769 INFO  (qtp990030861-50810) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[13]} 0 0
   [junit4]   2> 810770 INFO  (qtp990030861-50809) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[14]} 0 0
   [junit4]   2> 810771 INFO  (qtp990030861-50811) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[15]} 0 0
   [junit4]   2> 810773 INFO  (qtp990030861-50812) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[16]} 0 0
   [junit4]   2> 810774 INFO  (qtp990030861-50805) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[17]} 0 0
   [junit4]   2> 810776 INFO  (qtp990030861-50808) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[18]} 0 0
   [junit4]   2> 810777 INFO  (qtp990030861-50810) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[19]} 0 0
   [junit4]   2> 810779 INFO  (qtp990030861-50809) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[20]} 0 0
   [junit4]   2> 810780 INFO  (qtp990030861-50811) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[21]} 0 0
   [junit4]   2> 810781 INFO  (qtp990030861-50812) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[22]} 0 0
   [junit4]   2> 810783 INFO  (qtp990030861-50805) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[23]} 0 0
   [junit4]   2> 810784 INFO  (qtp990030861-50808) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[24]} 0 0
   [junit4]   2> 810786 INFO  (qtp990030861-50810) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[25]} 0 0
   [junit4]   2> 810787 INFO  (qtp990030861-50810) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[26]} 0 0
   [junit4]   2> 810789 INFO  (qtp990030861-50810) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[27]} 0 0
   [junit4]   2> 810790 INFO  (qtp990030861-50810) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[28]} 0 0
   [junit4]   2> 810792 INFO  (qtp990030861-50805) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[29]} 0 0
   [junit4]   2> 810793 INFO  (qtp990030861-50808) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[30]} 0 0
   [junit4]   2> 810795 INFO  (qtp990030861-50809) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[31]} 0 0
   [junit4]   2> 810798 INFO  (qtp990030861-50811) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[32]} 0 0
   [junit4]   2> 810799 INFO  (qtp990030861-50812) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[33]} 0 0
   [junit4]   2> 810801 INFO  (qtp990030861-50812) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[34]} 0 0
   [junit4]   2> 810802 INFO  (qtp990030861-50812) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[35]} 0 0
   [junit4]   2> 810803 INFO  (qtp990030861-50808) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[36]} 0 0
   [junit4]   2> 810805 INFO  (qtp990030861-50809) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[37]} 0 0
   [junit4]   2> 810806 INFO  (qtp990030861-50811) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[38]} 0 0
   [junit4]   2> 810808 INFO  (qtp990030861-50810) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[39]} 0 0
   [junit4]   2> 810809 INFO  (qtp990030861-50810) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[40]} 0 0
   [junit4]   2> 810810 INFO  (qtp990030861-50812) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[41]} 0 0
   [junit4]   2> 810812 INFO  (qtp990030861-50808) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[42]} 0 0
   [junit4]   2> 810813 INFO  (qtp990030861-50809) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[43]} 0 0
   [junit4]   2> 810815 INFO  (qtp990030861-50811) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[44]} 0 0
   [junit4]   2> 810816 INFO  (qtp990030861-50805) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[45]} 0 0
   [junit4]   2> 810818 INFO  (qtp990030861-50810) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[46]} 0 0
   [junit4]   2> 810819 INFO  (qtp990030861-50812) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[47]} 0 0
   [junit4]   2> 810821 INFO  (qtp990030861-50808) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[48]} 0 0
   [junit4]   2> 810822 INFO  (qtp990030861-50809) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[49]} 0 0
   [junit4]   2> 810824 INFO  (qtp990030861-50811) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[50]} 0 0
   [junit4]   2> 810825 INFO  (qtp990030861-50805) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[51]} 0 0
   [junit4]   2> 810826 INFO  (qtp990030861-50805) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[52]} 0 0
   [junit4]   2> 810828 INFO  (qtp990030861-50812) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[53]} 0 0
   [junit4]   2> 810829 INFO  (qtp990030861-50808) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[54]} 0 0
   [junit4]   2> 810831 INFO  (qtp990030861-50809) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[55]} 0 0
   [junit4]   2> 810832 INFO  (qtp990030861-50811) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[56]} 0 0
   [junit4]   2> 810834 INFO  (qtp990030861-50810) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[57]} 0 0
   [junit4]   2> 810835 INFO  (qtp990030861-50805) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[58]} 0 0
   [junit4]   2> 810837 INFO  (qtp990030861-50812) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[59]} 0 0
   [junit4]   2> 810838 INFO  (qtp990030861-50808) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[60]} 0 0
   [junit4]   2> 810839 INFO  (qtp990030861-50809) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[61]} 0 0
   [junit4]   2> 810841 INFO  (qtp990030861-50811) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[62]} 0 0
   [junit4]   2> 810842 INFO  (qtp990030861-50810) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[63]} 0 0
   [junit4]   2> 810844 INFO  (qtp990030861-50805) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[64]} 0 0
   [junit4]   2> 810845 INFO  (qtp990030861-50812) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[65]} 0 0
   [junit4]   2> 810847 INFO  (qtp990030861-50808) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[66]} 0 0
   [junit4]   2> 810848 INFO  (qtp990030861-50809) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[67]} 0 0
   [junit4]   2> 810850 INFO  (qtp990030861-50811) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[68]} 0 0
   [junit4]   2> 810851 INFO  (qtp990030861-50810) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[69]} 0 0
   [junit4]   2> 810853 INFO  (qtp990030861-50805) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[70]} 0 0
   [junit4]   2> 810855 INFO  (qtp990030861-50812) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[71]} 0 0
   [junit4]   2> 810856 INFO  (qtp990030861-50808) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[72]} 0 0
   [junit4]   2> 810858 INFO  (qtp990030861-50809) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[73]} 0 0
   [junit4]   2> 810859 INFO  (qtp990030861-50811) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[74]} 0 0
   [junit4]   2> 810860 INFO  (qtp990030861-50810) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[75]} 0 0
   [junit4]   2> 810862 INFO  (qtp990030861-50805) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[76]} 0 0
   [junit4]   2> 810863 INFO  (qtp990030861-50812) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[77]} 0 0
   [junit4]   2> 810865 INFO  (qtp990030861-50808) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[78]} 0 0
   [junit4]   2> 810866 INFO  (qtp990030861-50809) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[79]} 0 0
   [junit4]   2> 810868 INFO  (qtp990030861-50811) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[80]} 0 0
   [junit4]   2> 810869 INFO  (qtp990030861-50810) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[81]} 0 0
   [junit4]   2> 810871 INFO  (qtp990030861-50805) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[82]} 0 0
   [junit4]   2> 810872 INFO  (qtp990030861-50812) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[83]} 0 0
   [junit4]   2> 810874 INFO  (qtp990030861-50808) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[84]} 0 0
   [junit4]   2> 810875 INFO  (qtp990030861-50809) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[85]} 0 0
   [junit4]   2> 810877 INFO  (qtp990030861-50811) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[86]} 0 0
   [junit4]   2> 810878 INFO  (qtp990030861-50810) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[87]} 0 0
   [junit4]   2> 810880 INFO  (qtp990030861-50805) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[88]} 0 0
   [junit4]   2> 810881 INFO  (qtp990030861-50812) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[89]} 0 0
   [junit4]   2> 810883 INFO  (qtp990030861-50808) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[90]} 0 0
   [junit4]   2> 810884 INFO  (qtp990030861-50809) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[91]} 0 0
   [junit4]   2> 810886 INFO  (qtp990030861-50811) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[92]} 0 0
   [junit4]   2> 810887 INFO  (qtp990030861-50810) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[93]} 0 0
   [junit4]   2> 810889 INFO  (qtp990030861-50805) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[94]} 0 0
   [junit4]   2> 810890 INFO  (qtp990030861-50812) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[95]} 0 0
   [junit4]   2> 810892 INFO  (qtp990030861-50812) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[96]} 0 0
   [junit4]   2> 810893 INFO  (qtp990030861-50809) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[97]} 0 0
   [junit4]   2> 810894 INFO  (qtp990030861-50811)

[...truncated too long message...]

15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-subtree-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-trigger-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-jdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-jdbm1-2.0.0-M2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-kerberos-codec-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-ldif-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-mavibot-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-protocol-kerberos-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-protocol-ldap-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-protocol-shared-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-xdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/api-all-1.0.0-M20.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/bcprov-jdk15-1.45.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/cglib-nodep-2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/commons-collections-3.2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/commons-math3-3.4.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/easymock-3.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/ehcache-core-2.4.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/hadoop-common-2.7.2-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/hadoop-hdfs-2.7.2-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/hadoop-minikdc-2.7.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/htrace-core-3.2.0-incubating.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jackson-annotations-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jackson-core-asl-1.9.13.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jackson-databind-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jackson-mapper-asl-1.9.13.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jersey-core-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jersey-server-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jetty-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jetty-util-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/mina-core-2.0.0-M5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/mockito-core-1.9.5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/netty-all-4.0.36.Final.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/objenesis-1.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/presto-parser-0.122.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/slice-0.10.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/analysis/icu/lucene-analyzers-icu-6.2.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/contrib/solr-analysis-extras/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/queryparser/classes/test:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/contrib/analysis-extras/lib/icu4j-56.1.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-launcher.jar:/x1/jenkins/.ant/lib/ivy-2.3.0.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-antlr.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-regexp.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-resolver.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-net.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jmf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-oro.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-xalan2.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-logging.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jsch.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit4.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-javamail.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jai.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jdepend.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bsf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-swing.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-netrexx.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bcel.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-testutil.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-log4j.jar:/x1/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.8/lib/tools.jar:/x1/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.3.4.jar com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/temp/junit4-J2-20160805_152100_908.events @/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/temp/junit4-J2-20160805_152100_908.suites -stdin
   [junit4] ERROR: JVM J2 ended with an exception: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1501)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:121)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:955)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:952)
   [junit4] 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4] 	at java.lang.Thread.run(Thread.java:745)

BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:771: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:707: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/common-build.xml:536: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/common-build.xml:1443: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/common-build.xml:984: At least one slave process threw an exception, first: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.

Total time: 290 minutes 18 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
ERROR: Failed to archive {heapdumps/java_pid24995.hprof=heapdumps/java_pid24995.hprof, README.txt=README.txt, heapdumps/java_pid24984.hprof=heapdumps/java_pid24984.hprof, solr/build/solr-core/test/temp/junit4-J1-20160805_152100_908.events=solr/build/solr-core/test/temp/junit4-J1-20160805_152100_908.events, solr/build/solr-core/test/temp/junit4-J2-20160805_152100_908.events=solr/build/solr-core/test/temp/junit4-J2-20160805_152100_908.events, solr/build/solr-core/test/temp/junit4-J0-20160805_152100_908.events=solr/build/solr-core/test/temp/junit4-J0-20160805_152100_908.events} due to internal error; falling back to full archiving
java.lang.NullPointerException
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any




[JENKINS] Lucene-Solr-NightlyTests-6.x - Build # 136 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-6.x/136/

5 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.CollectionsAPIDistributedZkTest

Error Message:
1 thread leaked from SUITE scope at org.apache.solr.cloud.CollectionsAPIDistributedZkTest:     1) Thread[id=25792, name=searcherExecutor-8679-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.cloud.CollectionsAPIDistributedZkTest: 
   1) Thread[id=25792, name=searcherExecutor-8679-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
	at __randomizedtesting.SeedInfo.seed([447D65A31375D152]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.CollectionsAPIDistributedZkTest

Error Message:
There are still zombie threads that couldn't be terminated:    1) Thread[id=25792, name=searcherExecutor-8679-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   1) Thread[id=25792, name=searcherExecutor-8679-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
	at __randomizedtesting.SeedInfo.seed([447D65A31375D152]:0)


FAILED:  org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.test

Error Message:
Timeout waiting for all live and active

Stack Trace:
java.lang.AssertionError: Timeout waiting for all live and active
	at __randomizedtesting.SeedInfo.seed([447D65A31375D152:CC295A79BD89BCAA]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.testBasics(SharedFSAutoReplicaFailoverTest.java:309)
	at org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.test(SharedFSAutoReplicaFailoverTest.java:127)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=12467, name=collection1, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=12467, name=collection1, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:57130: collection already exists: awholynewstresscollection_collection1_3
	at __randomizedtesting.SeedInfo.seed([447D65A31375D152]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:592)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:261)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:250)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:403)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:356)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1291)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1061)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:997)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1599)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1620)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:987)


FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Timeout occured while waiting response from server at: https://127.0.0.1:40237/fl_x/t

Stack Trace:
org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: https://127.0.0.1:40237/fl_x/t
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:603)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:261)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:250)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.makeRequest(CollectionsAPIDistributedZkTest.java:399)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testErrorHandling(CollectionsAPIDistributedZkTest.java:457)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test(CollectionsAPIDistributedZkTest.java:179)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.net.SocketTimeoutException: Read timed out
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
	at java.net.SocketInputStream.read(SocketInputStream.java:170)
	at java.net.SocketInputStream.read(SocketInputStream.java:141)
	at sun.security.ssl.InputRecord.readFully(InputRecord.java:465)
	at sun.security.ssl.InputRecord.read(InputRecord.java:503)
	at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:973)
	at sun.security.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:930)
	at sun.security.ssl.AppInputStream.read(AppInputStream.java:105)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:160)
	at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:84)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:273)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
	at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
	at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)
	at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)
	at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197)
	at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)
	at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)
	at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685)
	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487)
	at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:497)
	... 47 more




Build Log:
[...truncated 11365 lines...]
   [junit4] Suite: org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_447D65A31375D152-001/init-core-data-001
   [junit4]   2> 254073 INFO  (SUITE-SharedFSAutoReplicaFailoverTest-seed#[447D65A31375D152]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=None)
   [junit4]   2> 254073 INFO  (SUITE-SharedFSAutoReplicaFailoverTest-seed#[447D65A31375D152]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /_fps/vg
   [junit4]   2> 254528 WARN  (SUITE-SharedFSAutoReplicaFailoverTest-seed#[447D65A31375D152]-worker) [    ] o.a.h.u.NativeCodeLoader Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
   [junit4]   1> Formatting using clusterid: testClusterID
   [junit4]   2> 255009 WARN  (SUITE-SharedFSAutoReplicaFailoverTest-seed#[447D65A31375D152]-worker) [    ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
   [junit4]   2> 255140 INFO  (SUITE-SharedFSAutoReplicaFailoverTest-seed#[447D65A31375D152]-worker) [    ] o.m.log Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
   [junit4]   2> 255161 WARN  (SUITE-SharedFSAutoReplicaFailoverTest-seed#[447D65A31375D152]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 255209 INFO  (SUITE-SharedFSAutoReplicaFailoverTest-seed#[447D65A31375D152]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 255242 INFO  (SUITE-SharedFSAutoReplicaFailoverTest-seed#[447D65A31375D152]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/hdfs to ./temp/Jetty_localhost_50023_hdfs____ek1uow/webapp
   [junit4]   2> 255891 INFO  (SUITE-SharedFSAutoReplicaFailoverTest-seed#[447D65A31375D152]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:50023
   [junit4]   2> 256456 WARN  (SUITE-SharedFSAutoReplicaFailoverTest-seed#[447D65A31375D152]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 256460 INFO  (SUITE-SharedFSAutoReplicaFailoverTest-seed#[447D65A31375D152]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 256471 INFO  (SUITE-SharedFSAutoReplicaFailoverTest-seed#[447D65A31375D152]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_51821_datanode____.84rgou/webapp
   [junit4]   2> 256819 INFO  (SUITE-SharedFSAutoReplicaFailoverTest-seed#[447D65A31375D152]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:51821
   [junit4]   2> 257081 WARN  (SUITE-SharedFSAutoReplicaFailoverTest-seed#[447D65A31375D152]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 257082 INFO  (SUITE-SharedFSAutoReplicaFailoverTest-seed#[447D65A31375D152]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 257091 INFO  (SUITE-SharedFSAutoReplicaFailoverTest-seed#[447D65A31375D152]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_32782_datanode____rtwux/webapp
   [junit4]   2> 257494 INFO  (SUITE-SharedFSAutoReplicaFailoverTest-seed#[447D65A31375D152]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:32782
   [junit4]   2> 258136 INFO  (IPC Server handler 8 on 37901) [    ] BlockStateChange BLOCK* processReport: from storage DS-a8f5c190-63b3-4629-807f-fc05a04e0d4b node DatanodeRegistration(127.0.0.1:58374, datanodeUuid=80f943f2-4820-44f7-bf0e-2f90fe38385e, infoPort=45592, infoSecurePort=0, ipcPort=42985, storageInfo=lv=-56;cid=testClusterID;nsid=745202128;c=0), blocks: 0, hasStaleStorage: true, processing time: 10 msecs
   [junit4]   2> 258136 INFO  (IPC Server handler 8 on 37901) [    ] BlockStateChange BLOCK* processReport: from storage DS-6453c9b3-e4b7-46f6-bc25-0d3bd7b2f588 node DatanodeRegistration(127.0.0.1:58374, datanodeUuid=80f943f2-4820-44f7-bf0e-2f90fe38385e, infoPort=45592, infoSecurePort=0, ipcPort=42985, storageInfo=lv=-56;cid=testClusterID;nsid=745202128;c=0), blocks: 0, hasStaleStorage: false, processing time: 0 msecs
   [junit4]   2> 258143 INFO  (IPC Server handler 9 on 37901) [    ] BlockStateChange BLOCK* processReport: from storage DS-6bbbc032-999f-4e07-9db6-4db3190e8013 node DatanodeRegistration(127.0.0.1:46885, datanodeUuid=be6f71d1-6f3a-4256-9a8b-1400ad64c176, infoPort=54972, infoSecurePort=0, ipcPort=60610, storageInfo=lv=-56;cid=testClusterID;nsid=745202128;c=0), blocks: 0, hasStaleStorage: true, processing time: 0 msecs
   [junit4]   2> 258143 INFO  (IPC Server handler 9 on 37901) [    ] BlockStateChange BLOCK* processReport: from storage DS-f0b2f890-22e8-4534-91c2-2b3547b5cddd node DatanodeRegistration(127.0.0.1:46885, datanodeUuid=be6f71d1-6f3a-4256-9a8b-1400ad64c176, infoPort=54972, infoSecurePort=0, ipcPort=60610, storageInfo=lv=-56;cid=testClusterID;nsid=745202128;c=0), blocks: 0, hasStaleStorage: false, processing time: 0 msecs
   [junit4]   2> 258443 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 258443 INFO  (Thread-834) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 258443 INFO  (Thread-834) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 258543 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.ZkTestServer start zk server on port:44085
   [junit4]   2> 258544 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 258544 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 258547 INFO  (zkCallback-521-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1f670e2 name:ZooKeeperConnection Watcher:127.0.0.1:44085 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 258547 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 258547 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 258547 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 258550 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 258550 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 258551 INFO  (zkCallback-522-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2cd37c90 name:ZooKeeperConnection Watcher:127.0.0.1:44085/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 258551 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 258551 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 258551 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 258559 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 258562 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 258563 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 258563 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 258564 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 258565 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
   [junit4]   2> 258565 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 258566 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 258566 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 258567 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 258567 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 258568 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 258568 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 258569 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 258569 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 258570 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 258570 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 258570 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 258570 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 258571 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 258571 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 258572 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 258572 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 258573 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 258573 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 258634 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_447D65A31375D152-001/control-001/cores/collection1
   [junit4]   2> 258635 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.e.j.s.Server jetty-9.3.8.v20160314
   [junit4]   2> 258636 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@56d4dd3b{/_fps/vg,null,AVAILABLE}
   [junit4]   2> 258636 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.e.j.s.ServerConnector Started ServerConnector@60ada1ca{HTTP/1.1,[http/1.1]}{127.0.0.1:53020}
   [junit4]   2> 258636 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.e.j.s.Server Started @260903ms
   [junit4]   2> 258636 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/_fps/vg, hostPort=53020, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_447D65A31375D152-001/control-001/cores}
   [junit4]   2> 258637 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@73d16e93
   [junit4]   2> 258637 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_447D65A31375D152-001/control-001'
   [junit4]   2> 258637 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 258637 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 258637 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 258637 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 258639 INFO  (zkCallback-523-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@470f3bf7 name:ZooKeeperConnection Watcher:127.0.0.1:44085/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 258639 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 258639 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 258640 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 258640 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_447D65A31375D152-001/control-001/solr.xml
   [junit4]   2> 258644 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_447D65A31375D152-001/control-001/cores
   [junit4]   2> 258644 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.CoreContainer New CoreContainer 1942926815
   [junit4]   2> 258644 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_447D65A31375D152-001/control-001]
   [junit4]   2> 258644 WARN  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_447D65A31375D152-001/control-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_447D65A31375D152-001/control-001/lib
   [junit4]   2> 258644 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,connectionsEvictorSleepDelay : 5000,maxConnectionIdleTime : 40000,
   [junit4]   2> 258655 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 258656 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 258656 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 258656 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:44085/solr
   [junit4]   2> 258656 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 258656 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 258657 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 258657 INFO  (zkCallback-526-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@53cd312e name:ZooKeeperConnection Watcher:127.0.0.1:44085 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 258658 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 258658 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 258659 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [n:127.0.0.1:53020__fps%2Fvg    ] o.a.s.c.ZkController Added new OnReconnect listener org.apache.solr.cloud.ZkController$$Lambda$23/1149697902@4ab3c19d
   [junit4]   2> 258660 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [n:127.0.0.1:53020__fps%2Fvg    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 258660 INFO  (zkCallback-527-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@10e10a8c name:ZooKeeperConnection Watcher:127.0.0.1:44085/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 258660 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [n:127.0.0.1:53020__fps%2Fvg    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 258661 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [n:127.0.0.1:53020__fps%2Fvg    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 258661 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [n:127.0.0.1:53020__fps%2Fvg    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 258662 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [n:127.0.0.1:53020__fps%2Fvg    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 258663 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [n:127.0.0.1:53020__fps%2Fvg    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 258664 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [n:127.0.0.1:53020__fps%2Fvg    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 258665 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [n:127.0.0.1:53020__fps%2Fvg    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 258665 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [n:127.0.0.1:53020__fps%2Fvg    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 258666 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [n:127.0.0.1:53020__fps%2Fvg    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 258666 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [n:127.0.0.1:53020__fps%2Fvg    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 258667 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [n:127.0.0.1:53020__fps%2Fvg    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 258667 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [n:127.0.0.1:53020__fps%2Fvg    ] o.a.s.c.c.ZkStateReader Loaded empty cluster properties
   [junit4]   2> 258667 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [n:127.0.0.1:53020__fps%2Fvg    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (0)
   [junit4]   2> 258668 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [n:127.0.0.1:53020__fps%2Fvg    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 258669 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [n:127.0.0.1:53020__fps%2Fvg    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 258671 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [n:127.0.0.1:53020__fps%2Fvg    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 258671 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [n:127.0.0.1:53020__fps%2Fvg    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/96358146939879428-127.0.0.1:53020__fps%2Fvg-n_0000000000
   [junit4]   2> 258672 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [n:127.0.0.1:53020__fps%2Fvg    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:53020__fps%2Fvg
   [junit4]   2> 258672 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [n:127.0.0.1:53020__fps%2Fvg    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 258672 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [n:127.0.0.1:53020__fps%2Fvg    ] o.a.s.c.Overseer Overseer (id=96358146939879428-127.0.0.1:53020__fps%2Fvg-n_0000000000) starting
   [junit4]   2> 258673 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [n:127.0.0.1:53020__fps%2Fvg    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 258676 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [n:127.0.0.1:53020__fps%2Fvg    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 258677 INFO  (OverseerStateUpdate-96358146939879428-127.0.0.1:53020__fps%2Fvg-n_0000000000) [n:127.0.0.1:53020__fps%2Fvg    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 258677 INFO  (OverseerCollectionConfigSetProcessor-96358146939879428-127.0.0.1:53020__fps%2Fvg-n_0000000000) [n:127.0.0.1:53020__fps%2Fvg    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 258680 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [n:127.0.0.1:53020__fps%2Fvg    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:53020__fps%2Fvg
   [junit4]   2> 258680 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [n:127.0.0.1:53020__fps%2Fvg    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:53020__fps%2Fvg
   [junit4]   2> 258681 INFO  (OverseerStateUpdate-96358146939879428-127.0.0.1:53020__fps%2Fvg-n_0000000000) [n:127.0.0.1:53020__fps%2Fvg    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (0)
   [junit4]   2> 258682 INFO  (zkCallback-527-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [0])
   [junit4]   2> 258682 INFO  (zkCallback-527-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 258691 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [n:127.0.0.1:53020__fps%2Fvg    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 258691 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [n:127.0.0.1:53020__fps%2Fvg    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 258691 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [n:127.0.0.1:53020__fps%2Fvg    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_447D65A31375D152-001/control-001/cores
   [junit4]   2> 258692 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [n:127.0.0.1:53020__fps%2Fvg    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, loadOnStartup=true, configSetProperties=configsetprops.json, collection=control_collection, coreNodeName=, dataDir=data/, shard=}
   [junit4]   2> 258692 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [n:127.0.0.1:53020__fps%2Fvg    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_447D65A31375D152-001/control-001/cores/collection1
   [junit4]   2> 258692 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [n:127.0.0.1:53020__fps%2Fvg    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 258693 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 258693 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 258693 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [n:127.0.0.1:53020__fps%2Fvg    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1
   [junit4]   2> 258693 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [n:127.0.0.1:53020__fps%2Fvg    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 258693 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection   x:collection1] o.a.s.c.ZkController Collection control_collection not visible yet, but flagging it so a watch is registered when it becomes visible
   [junit4]   2> 258693 INFO  (OverseerStateUpdate-96358146939879428-127.0.0.1:53020__fps%2Fvg-n_0000000000) [n:127.0.0.1:53020__fps%2Fvg    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:53020/_fps/vg",
   [junit4]   2>   "node_name":"127.0.0.1:53020__fps%2Fvg",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 258694 INFO  (OverseerStateUpdate-96358146939879428-127.0.0.1:53020__fps%2Fvg-n_0000000000) [n:127.0.0.1:53020__fps%2Fvg    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:53020/_fps/vg",
   [junit4]   2>   "node_name":"127.0.0.1:53020__fps%2Fvg",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 258694 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Deleting data for [control_collection]
   [junit4]   2> 258694 INFO  (OverseerStateUpdate-96358146939879428-127.0.0.1:53020__fps%2Fvg-n_0000000000) [n:127.0.0.1:53020__fps%2Fvg    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 258694 INFO  (OverseerStateUpdate-96358146939879428-127.0.0.1:53020__fps%2Fvg-n_0000000000) [n:127.0.0.1:53020__fps%2Fvg    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 258695 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 258697 INFO  (zkCallback-527-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 259695 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 259695 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 259695 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 259695 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/control_collection]
   [junit4]   2> 259696 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/control_collection] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 259696 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_447D65A31375D152-001/control-001/cores/collection1'
   [junit4]   2> 259696 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 259696 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 259697 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 259702 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 259705 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.2.0
   [junit4]   2> 259712 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection   x:collection1] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 259715 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 259797 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 259798 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 259798 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 259800 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 259817 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1
   [junit4]   2> 259818 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 259818 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.HdfsDirectoryFactory
   [junit4]   2> 259825 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:37901/solr_hdfs_home
   [junit4]   2> 259825 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
   [junit4]   2> 259825 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_447D65A31375D152-001/control-001/cores/collection1], dataDir=[hdfs://localhost:37901/solr_hdfs_home/control_collection/core_node1/data/]
   [junit4]   2> 259825 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@3ee102d8
   [junit4]   2> 259825 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:37901/solr_hdfs_home/control_collection/core_node1/data/snapshot_metadata
   [junit4]   2> 259838 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
   [junit4]   2> 259838 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
   [junit4]   2> 259839 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
   [junit4]   2> 261722 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
   [junit4]   2> 261722 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:37901/solr_hdfs_home/control_collection/core_node1/data/snapshot_metadata
   [junit4]   2> 261723 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.s.SolrSnapshotMetaDataManager Loading from snapshot metadata file...
   [junit4]   2> 261728 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:37901/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 261742 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:37901/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 261744 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=hdfs://localhost:37901/solr_hdfs_home/control_collection/core_node1/data/index/
   [junit4]   2> 261758 WARN  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory 'hdfs:/localhost:37901/solr_hdfs_home/control_collection/core_node1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 261758 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:37901/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 261764 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
   [junit4]   2> 261764 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
   [junit4]   2> 261764 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
   [junit4]   2> 261774 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
   [junit4]   2> 261774 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:37901/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 261775 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=26, maxMergeAtOnceExplicit=43, maxMergedSegmentMB=87.306640625, floorSegmentMB=1.6142578125, forceMergeDeletesPctAllowed=1.726215935344575, segmentsPerTier=23.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 261974 INFO  (IPC Server handler 1 on 37901) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:58374 is added to blk_1073741825_1001{UCState=COMMITTED, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-6453c9b3-e4b7-46f6-bc25-0d3bd7b2f588:NORMAL:127.0.0.1:58374|RBW], ReplicaUC[[DISK]DS-6bbbc032-999f-4e07-9db6-4db3190e8013:NORMAL:127.0.0.1:46885|RBW]]} size 71
   [junit4]   2> 261975 INFO  (IPC Server handler 6 on 37901) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:46885 is added to blk_1073741825_1001 size 71
   [junit4]   2> 262384 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(BlockDirectory(HdfsDirectory@hdfs://localhost:37901/solr_hdfs_home/control_collection/core_node1/data/index lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@4d881e82); maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 262384 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 262386 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in hdfs://localhost:37901/solr_hdfs_home/control_collection/core_node1/data/
   [junit4]   2> 262388 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 262388 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 262388 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 262388 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 262388 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 262389 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 262389 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 262389 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 262389 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 262389 WARN  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
   [junit4]   2> 262390 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 262391 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 262391 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 262392 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 262401 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/mbeans,standard,/update/json/docs,/admin/luke,/export,/get,/admin/properties,/update/json,/admin/threads,/dump,/update/csv,/sql,/graph,/admin/segments,/admin/system,/replication,/config,/stream,/schema,/admin/plugins,/admin/logging,/admin/ping,/update,/admin/file,/terms
   [junit4]   2> 262401 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 262410 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
   [junit4]   2> 262410 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 262410 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=2
   [junit4]   2> 262421 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 262421 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 262423 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@35f5b7de
   [junit4]   2> 262479 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(BlockDirectory(HdfsDirectory@hdfs://localhost:37901/solr_hdfs_home/control_collection/core_node1/data/index lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@4d881e82); maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 262479 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 262479 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@3466b80[collection1] main]
   [junit4]   2> 262479 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/control_collection]
   [junit4]   2> 262480 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/control_collection] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 262480 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 262480 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 262480 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 262480 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 262480 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 262480 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 262480 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 262482 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 262483 INFO  (searcherExecutor-986-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@3466b80[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 262483 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 262483 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 262484 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 262484 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1541730355273269248
   [junit4]   2> 262485 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 1.0ms to seed version buckets with highest version 1541730355273269248
   [junit4]   2> 262485 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 262486 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 262486 INFO  (coreZkRegister-979-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:53020/_fps/vg collection:control_collection shard:shard1
   [junit4]   2> 262487 INFO  (coreZkRegister-979-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 262489 INFO  (coreZkRegister-979-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/96358146939879428-core_node1-n_0000000000
   [junit4]   2> 262489 INFO  (coreZkRegister-979-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard=shard1 and weAreReplacement=false and leaderVoteWait=10000
   [junit4]   2> 262490 INFO  (coreZkRegister-979-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 262490 INFO  (OverseerStateUpdate-96358146939879428-127.0.0.1:53020__fps%2Fvg-n_0000000000) [n:127.0.0.1:53020__fps%2Fvg    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 262490 INFO  (coreZkRegister-979-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 262490 INFO  (coreZkRegister-979-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:53020/_fps/vg/collection1/
   [junit4]   2> 262491 INFO  (coreZkRegister-979-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 262491 INFO  (coreZkRegister-979-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:53020/_fps/vg/collection1/ has no replicas
   [junit4]   2> 262491 INFO  (zkCallback-527-thread-2-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 262492 INFO  (coreZkRegister-979-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 262493 INFO  (coreZkRegister-979-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/control_collection/leaders/shard1/leader after winning as /collections/control_collection/leader_elect/shard1/election/96358146939879428-core_node1-n_0000000000
   [junit4]   2> 262494 INFO  (coreZkRegister-979-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:53020/_fps/vg/collection1/ shard1
   [junit4]   2> 262495 INFO  (OverseerStateUpdate-96358146939879428-127.0.0.1:53020__fps%2Fvg-n_0000000000) [n:127.0.0.1:53020__fps%2Fvg    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:53020/_fps/vg",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 2
   [junit4]   2> 262596 INFO  (zkCallback-527-thread-2-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 262643 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 262643 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 262644 INFO  (zkCallback-531-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@263f1b79 name:ZooKeeperConnection Watcher:127.0.0.1:44085/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 262644 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 262644 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 262644 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 262645 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.c.ZkStateReader Loaded empty cluster properties
   [junit4]   2> 262645 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 262646 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 262646 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with stateFormat=2
   [junit4]   2> 262646 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 262646 INFO  (coreZkRegister-979-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:53020/_fps/vg/collection1/ and leader is http://127.0.0.1:53020/_fps/vg/collection1/
   [junit4]   2> 262646 INFO  (coreZkRegister-979-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:53020/_fps/vg
   [junit4]   2> 262646 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 262646 INFO  (coreZkRegister-979-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 262646 INFO  (coreZkRegister-979-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 262646 INFO  (coreZkRegister-979-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 262647 INFO  (coreZkRegister-979-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Checking legacy cluster state for collection control_collection
   [junit4]   2> 262647 INFO  (OverseerStateUpdate-96358146939879428-127.0.0.1:53020__fps%2Fvg-n_0000000000) [n:127.0.0.1:53020__fps%2Fvg    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "dataDir":"hdfs://localhost:37901/solr_hdfs_home/control_collection/core_node1/data/",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:53020/_fps/vg",
   [junit4]   2>   "node_name":"127.0.0.1:53020__fps%2Fvg",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "ulogDir":"hdfs://localhost:37901/solr_hdfs_home/control_collection/core_node1/data/tlog"} current state version: 3
   [junit4]   2> 262648 INFO  (OverseerStateUpdate-96358146939879428-127.0.0.1:53020__fps%2Fvg-n_0000000000) [n:127.0.0.1:53020__fps%2Fvg    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "dataDir":"hdfs://localhost:37901/solr_hdfs_home/control_collection/core_node1/data/",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:5302

[...truncated too long message...]

tedZkTest-seed#[447D65A31375D152]-worker) [    ] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_447D65A31375D152-001
   [junit4]   2> Aug 04, 2016 12:11:38 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 2 leaked thread(s).
   [junit4]   2> Aug 04, 2016 12:11:58 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> SEVERE: 1 thread leaked from SUITE scope at org.apache.solr.cloud.CollectionsAPIDistributedZkTest: 
   [junit4]   2>    1) Thread[id=25792, name=searcherExecutor-8679-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]   2>         at sun.misc.Unsafe.park(Native Method)
   [junit4]   2>         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
   [junit4]   2>         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
   [junit4]   2>         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]   2>         at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> Aug 04, 2016 12:11:58 PM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
   [junit4]   2> INFO: Starting to interrupt leaked threads:
   [junit4]   2>    1) Thread[id=25792, name=searcherExecutor-8679-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]   2> Aug 04, 2016 12:12:01 PM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
   [junit4]   2> SEVERE: There are still zombie threads that couldn't be terminated:
   [junit4]   2>    1) Thread[id=25792, name=searcherExecutor-8679-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]   2>         at sun.misc.Unsafe.park(Native Method)
   [junit4]   2>         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
   [junit4]   2>         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
   [junit4]   2>         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]   2>         at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene62), sim=RandomSimilarity(queryNorm=false,coord=crazy): {}, locale=vi-VN, timezone=Mexico/BajaNorte
   [junit4]   2> NOTE: Linux 3.13.0-85-generic amd64/Oracle Corporation 1.8.0_74 (64-bit)/cpus=4,threads=2,free=149933408,total=495976448
   [junit4]   2> NOTE: All tests run in this JVM: [ZkStateReaderTest, ShardRoutingTest, ChaosMonkeySafeLeaderTest, IndexBasedSpellCheckerTest, WordBreakSolrSpellCheckerTest, ChaosMonkeyNothingIsSafeTest, DistanceUnitsTest, TermVectorComponentTest, TestReplicationHandlerBackup, MBeansHandlerTest, TestUpdate, ZkControllerTest, SharedFSAutoReplicaFailoverTest, DistributedDebugComponentTest, SuggesterFSTTest, IgnoreCommitOptimizeUpdateProcessorFactoryTest, AsyncMigrateRouteKeyTest, TestNamedUpdateProcessors, LeaderInitiatedRecoveryOnShardRestartTest, OverseerModifyCollectionTest, TestSolrXml, TestCursorMarkWithoutUniqueKey, TestMacros, DistribCursorPagingTest, TestCodecSupport, OutOfBoxZkACLAndCredentialsProvidersTest, TestFieldSortValues, ClassificationUpdateProcessorFactoryTest, TestStressUserVersions, InfoHandlerTest, TestBM25SimilarityFactory, TestFiltering, TestNumericTerms64, TimeZoneUtilsTest, TestManagedSchemaAPI, TestHashQParserPlugin, DocValuesTest, TestSearcherReuse, SolrCmdDistributorTest, TestCryptoKeys, TestQuerySenderNoQuery, SolrIndexSplitterTest, SpatialFilterTest, TestMiniSolrCloudClusterSSL, TestHashPartitioner, DOMUtilTest, ReplicationFactorTest, TriLevelCompositeIdRoutingTest, CustomCollectionTest, TestSchemaSimilarityResource, TermsComponentTest, TestJsonRequest, TestPartialUpdateDeduplication, HdfsDirectoryTest, DistributedMLTComponentTest, BlockCacheTest, HdfsRecoveryZkTest, UniqFieldsUpdateProcessorFactoryTest, DirectSolrSpellCheckerTest, TestOrdValues, TestBulkSchemaAPI, TestSolrCoreProperties, FastVectorHighlighterTest, HdfsCollectionsAPIDistributedZkTest, TestXmlQParserPlugin, TestGraphMLResponseWriter, NotRequiredUniqueKeyTest, TestShardHandlerFactory, TestRandomRequestDistribution, TestConfigSetImmutable, TestExclusionRuleCollectionAccess, TestBulkSchemaConcurrent, TestCoreContainer, TestLFUCache, TestRequestForwarding, TestTrie, TestFastLRUCache, TestCollapseQParserPlugin, TestSolrQueryResponse, MoreLikeThisHandlerTest, SuggestComponentContextFilterQueryTest, TestMiniSolrCloudCluster, CoreAdminCreateDiscoverTest, CSVRequestHandlerTest, TestQueryUtils, TestExactStatsCache, TestSubQueryTransformer, FullHLLTest, HdfsWriteToMultipleCollectionsTest, TestDistributedGrouping, OpenCloseCoreStressTest, HttpPartitionTest, HighlighterTest, MultiTermTest, TestLeaderInitiatedRecoveryThread, TestAuthenticationFramework, ConvertedLegacyTest, SpellingQueryConverterTest, TestFieldCollectionResource, CollectionsAPIDistributedZkTest]
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=CollectionsAPIDistributedZkTest -Dtests.seed=447D65A31375D152 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=vi-VN -Dtests.timezone=Mexico/BajaNorte -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR   0.00s J1 | CollectionsAPIDistributedZkTest (suite) <<<
   [junit4]    > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.cloud.CollectionsAPIDistributedZkTest: 
   [junit4]    >    1) Thread[id=25792, name=searcherExecutor-8679-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
   [junit4]    >         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([447D65A31375D152]:0)Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   [junit4]    >    1) Thread[id=25792, name=searcherExecutor-8679-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
   [junit4]    >         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([447D65A31375D152]:0)
   [junit4] Completed [490/626 (3!)] on J1 in 619.04s, 1 test, 3 errors <<< FAILURES!

[...truncated 404 lines...]
   [junit4] JVM J1: stdout was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/temp/junit4-J1-20160804_105937_505.sysout
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] java.lang.OutOfMemoryError: GC overhead limit exceeded
   [junit4] Dumping heap to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/heapdumps/java_pid1205.hprof ...
   [junit4] Heap dump file created [685134994 bytes in 4.170 secs]
   [junit4] <<< JVM J1: EOF ----

[...truncated 11057 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:771: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:715: Some of the tests produced a heap dump, but did not fail. Maybe a suppressed OutOfMemoryError? Dumps created:
* java_pid1205.hprof

Total time: 237 minutes 9 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any




[JENKINS] Lucene-Solr-NightlyTests-6.x - Build # 135 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-6.x/135/

1 tests failed.
FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Timeout occured while waiting response from server at: http://127.0.0.1:46685

Stack Trace:
org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:46685
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:603)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:261)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:250)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.makeRequest(CollectionsAPIDistributedZkTest.java:399)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testErrorHandling(CollectionsAPIDistributedZkTest.java:457)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test(CollectionsAPIDistributedZkTest.java:179)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.net.SocketTimeoutException: Read timed out
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
	at java.net.SocketInputStream.read(SocketInputStream.java:170)
	at java.net.SocketInputStream.read(SocketInputStream.java:141)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:160)
	at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:84)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:273)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
	at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
	at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)
	at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)
	at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197)
	at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)
	at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)
	at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685)
	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487)
	at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:497)
	... 47 more




Build Log:
[...truncated 11617 lines...]
   [junit4] JVM J1: stdout was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/temp/junit4-J1-20160803_161425_766.sysout
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] java.lang.OutOfMemoryError: GC overhead limit exceeded
   [junit4] Dumping heap to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/heapdumps/java_pid1102.hprof ...
   [junit4] Heap dump file created [664574313 bytes in 4.467 secs]
   [junit4] <<< JVM J1: EOF ----

   [junit4] JVM J1: stderr was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/temp/junit4-J1-20160803_161425_766.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] WARN: Unhandled exception in event serialization. -> java.lang.OutOfMemoryError: GC overhead limit exceeded
   [junit4] <<< JVM J1: EOF ----

[...truncated 288 lines...]
   [junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_146D5B52C5AF51A-001/init-core-data-001
   [junit4]   2> 2723169 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[146D5B52C5AF51A]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
   [junit4]   2> 2723169 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[146D5B52C5AF51A]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   2> 2723171 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 2723171 INFO  (Thread-38431) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 2723171 INFO  (Thread-38431) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 2723271 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.ZkTestServer start zk server on port:50426
   [junit4]   2> 2723271 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2723271 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2723273 INFO  (zkCallback-2772-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@69a43639 name:ZooKeeperConnection Watcher:127.0.0.1:50426 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2723273 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2723274 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2723274 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 2723275 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2723276 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2723277 INFO  (zkCallback-2773-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1eb9e519 name:ZooKeeperConnection Watcher:127.0.0.1:50426/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2723277 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2723278 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2723278 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 2723279 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 2723280 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 2723280 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 2723281 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 2723281 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 2723282 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 2723282 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 2723283 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 2723283 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 2723284 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 2723284 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 2723285 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 2723285 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 2723286 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 2723286 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 2723287 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 2723287 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 2723287 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 2723287 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 2723288 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 2723288 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 2723289 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 2723289 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 2723290 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 2723290 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 2723348 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_146D5B52C5AF51A-001/control-001/cores/collection1
   [junit4]   2> 2723349 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.e.j.s.Server jetty-9.3.8.v20160314
   [junit4]   2> 2723350 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@4cc6748e{/,null,AVAILABLE}
   [junit4]   2> 2723353 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.e.j.s.ServerConnector Started ServerConnector@62a9519c{HTTP/1.1,[http/1.1]}{127.0.0.1:33120}
   [junit4]   2> 2723353 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.e.j.s.Server Started @2725578ms
   [junit4]   2> 2723353 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_146D5B52C5AF51A-001/tempDir-001/control/data, hostContext=/, hostPort=33120, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_146D5B52C5AF51A-001/control-001/cores}
   [junit4]   2> 2723353 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@73d16e93
   [junit4]   2> 2723353 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_146D5B52C5AF51A-001/control-001'
   [junit4]   2> 2723353 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 2723354 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 2723354 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2723354 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2723355 INFO  (zkCallback-2774-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2bebb18b name:ZooKeeperConnection Watcher:127.0.0.1:50426/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2723355 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2723355 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2723356 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 2723356 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_146D5B52C5AF51A-001/control-001/solr.xml
   [junit4]   2> 2723362 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_146D5B52C5AF51A-001/control-001/cores
   [junit4]   2> 2723362 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.CoreContainer New CoreContainer 1803722666
   [junit4]   2> 2723362 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_146D5B52C5AF51A-001/control-001]
   [junit4]   2> 2723362 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_146D5B52C5AF51A-001/control-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_146D5B52C5AF51A-001/control-001/lib
   [junit4]   2> 2723363 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,connectionsEvictorSleepDelay : 5000,maxConnectionIdleTime : 40000,
   [junit4]   2> 2723363 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 2723364 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 2723364 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 2723364 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:50426/solr
   [junit4]   2> 2723364 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 2723364 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2723364 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2723365 INFO  (zkCallback-2777-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@a735bd6 name:ZooKeeperConnection Watcher:127.0.0.1:50426 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2723365 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2723366 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2723366 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [n:127.0.0.1:33120_    ] o.a.s.c.ZkController Added new OnReconnect listener org.apache.solr.cloud.ZkController$$Lambda$12/976932739@70189276
   [junit4]   2> 2723366 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [n:127.0.0.1:33120_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2723367 INFO  (zkCallback-2778-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@643fd4c7 name:ZooKeeperConnection Watcher:127.0.0.1:50426/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2723367 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [n:127.0.0.1:33120_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2723367 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [n:127.0.0.1:33120_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 2723368 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [n:127.0.0.1:33120_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 2723369 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [n:127.0.0.1:33120_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 2723370 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [n:127.0.0.1:33120_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 2723371 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [n:127.0.0.1:33120_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 2723372 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [n:127.0.0.1:33120_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 2723372 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [n:127.0.0.1:33120_    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 2723373 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [n:127.0.0.1:33120_    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 2723373 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [n:127.0.0.1:33120_    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 2723374 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [n:127.0.0.1:33120_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 2723374 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [n:127.0.0.1:33120_    ] o.a.s.c.c.ZkStateReader Loaded empty cluster properties
   [junit4]   2> 2723374 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [n:127.0.0.1:33120_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (0)
   [junit4]   2> 2723376 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [n:127.0.0.1:33120_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 2723376 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [n:127.0.0.1:33120_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 2723377 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [n:127.0.0.1:33120_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 2723377 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [n:127.0.0.1:33120_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/96353883992948740-127.0.0.1:33120_-n_0000000000
   [junit4]   2> 2723377 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [n:127.0.0.1:33120_    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:33120_
   [junit4]   2> 2723377 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [n:127.0.0.1:33120_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 2723378 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [n:127.0.0.1:33120_    ] o.a.s.c.Overseer Overseer (id=96353883992948740-127.0.0.1:33120_-n_0000000000) starting
   [junit4]   2> 2723379 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [n:127.0.0.1:33120_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 2723381 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [n:127.0.0.1:33120_    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 2723382 INFO  (OverseerCollectionConfigSetProcessor-96353883992948740-127.0.0.1:33120_-n_0000000000) [n:127.0.0.1:33120_    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 2723382 INFO  (OverseerStateUpdate-96353883992948740-127.0.0.1:33120_-n_0000000000) [n:127.0.0.1:33120_    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 2723382 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [n:127.0.0.1:33120_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:33120_
   [junit4]   2> 2723382 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [n:127.0.0.1:33120_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:33120_
   [junit4]   2> 2723383 INFO  (OverseerStateUpdate-96353883992948740-127.0.0.1:33120_-n_0000000000) [n:127.0.0.1:33120_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (0)
   [junit4]   2> 2723383 INFO  (zkCallback-2778-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [0])
   [junit4]   2> 2723383 INFO  (zkCallback-2778-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 2723390 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [n:127.0.0.1:33120_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 2723390 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [n:127.0.0.1:33120_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 2723390 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [n:127.0.0.1:33120_    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_146D5B52C5AF51A-001/control-001/cores
   [junit4]   2> 2723391 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [n:127.0.0.1:33120_    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, loadOnStartup=true, configSetProperties=configsetprops.json, collection=control_collection, coreNodeName=, dataDir=data/, shard=}
   [junit4]   2> 2723391 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [n:127.0.0.1:33120_    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_146D5B52C5AF51A-001/control-001/cores/collection1
   [junit4]   2> 2723391 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [n:127.0.0.1:33120_    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 2723392 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 2723392 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [n:127.0.0.1:33120_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2
   [junit4]   2> 2723392 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 2723392 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [n:127.0.0.1:33120_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 2723397 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection   x:collection1] o.a.s.c.ZkController Collection control_collection not visible yet, but flagging it so a watch is registered when it becomes visible
   [junit4]   2> 2723397 INFO  (OverseerStateUpdate-96353883992948740-127.0.0.1:33120_-n_0000000000) [n:127.0.0.1:33120_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:33120",
   [junit4]   2>   "node_name":"127.0.0.1:33120_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 2723398 INFO  (OverseerStateUpdate-96353883992948740-127.0.0.1:33120_-n_0000000000) [n:127.0.0.1:33120_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:33120",
   [junit4]   2>   "node_name":"127.0.0.1:33120_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 2723398 INFO  (OverseerStateUpdate-96353883992948740-127.0.0.1:33120_-n_0000000000) [n:127.0.0.1:33120_    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 2723398 INFO  (OverseerStateUpdate-96353883992948740-127.0.0.1:33120_-n_0000000000) [n:127.0.0.1:33120_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 2723398 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Deleting data for [control_collection]
   [junit4]   2> 2723399 INFO  (zkCallback-2778-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 2723399 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 2723399 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 2723399 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 2723400 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 2723400 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/control_collection]
   [junit4]   2> 2723401 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/control_collection] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 2723401 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_146D5B52C5AF51A-001/control-001/cores/collection1'
   [junit4]   2> 2723401 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 2723401 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 2723402 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 2723407 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 2723410 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.2.0
   [junit4]   2> 2723416 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 2723419 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 2723486 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 2723490 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 2723491 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 2723496 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 2723497 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 2723498 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 2723499 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 2723500 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2
   [junit4]   2> 2723500 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 2723500 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 2723500 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_146D5B52C5AF51A-001/control-001/cores/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_146D5B52C5AF51A-001/control-001/cores/collection1/data/]
   [junit4]   2> 2723500 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@13518f49
   [junit4]   2> 2723501 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_146D5B52C5AF51A-001/control-001/cores/collection1/data/snapshot_metadata
   [junit4]   2> 2723501 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.s.SolrSnapshotMetaDataManager Loading from snapshot metadata file...
   [junit4]   2> 2723501 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_146D5B52C5AF51A-001/control-001/cores/collection1/data
   [junit4]   2> 2723501 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_146D5B52C5AF51A-001/control-001/cores/collection1/data/index/
   [junit4]   2> 2723501 WARN  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_146D5B52C5AF51A-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 2723502 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_146D5B52C5AF51A-001/control-001/cores/collection1/data/index
   [junit4]   2> 2723502 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=44, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
   [junit4]   2> 2723502 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@60326231 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@65454343),segFN=segments_1,generation=1}
   [junit4]   2> 2723502 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 2723502 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_146D5B52C5AF51A-001/control-001/cores/collection1/data/
   [junit4]   2> 2723502 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_146D5B52C5AF51A-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 2723505 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 2723505 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 2723505 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 2723505 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 2723505 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 2723505 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 2723505 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 2723505 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 2723505 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 2723505 WARN  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
   [junit4]   2> 2723506 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2723507 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2723507 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2723508 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2723509 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/mbeans,standard,/update/json/docs,/admin/luke,/export,/get,/admin/properties,/update/json,/admin/threads,/dump,/update/csv,/sql,/graph,/admin/segments,/admin/system,/replication,/config,/stream,/schema,/admin/plugins,/admin/logging,/admin/ping,/update,/admin/file,/terms
   [junit4]   2> 2723509 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 2723510 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 2723510 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 2723510 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 2723510 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 2723511 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@6af49be4
   [junit4]   2> 2723511 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@60326231 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@65454343),segFN=segments_1,generation=1}
   [junit4]   2> 2723511 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 2723511 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@bd7b90a[collection1] main]
   [junit4]   2> 2723511 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/control_collection]
   [junit4]   2> 2723512 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/control_collection] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 2723512 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 2723512 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 2723512 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 2723512 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 2723512 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 2723512 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 2723512 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 2723513 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 2723513 INFO  (searcherExecutor-5310-thread-1-processing-n:127.0.0.1:33120_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@bd7b90a[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2723513 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 2723513 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 2723513 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 2723514 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1541662144246841344
   [junit4]   2> 2723515 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 1.0ms to seed version buckets with highest version 1541662144246841344
   [junit4]   2> 2723515 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 2723516 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 2723516 INFO  (coreZkRegister-5303-thread-1-processing-n:127.0.0.1:33120_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:33120 collection:control_collection shard:shard1
   [junit4]   2> 2723517 INFO  (coreZkRegister-5303-thread-1-processing-n:127.0.0.1:33120_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 2723519 INFO  (coreZkRegister-5303-thread-1-processing-n:127.0.0.1:33120_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/96353883992948740-core_node1-n_0000000000
   [junit4]   2> 2723520 INFO  (coreZkRegister-5303-thread-1-processing-n:127.0.0.1:33120_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard=shard1 and weAreReplacement=false and leaderVoteWait=10000
   [junit4]   2> 2723521 INFO  (coreZkRegister-5303-thread-1-processing-n:127.0.0.1:33120_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 2723521 INFO  (coreZkRegister-5303-thread-1-processing-n:127.0.0.1:33120_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 2723521 INFO  (coreZkRegister-5303-thread-1-processing-n:127.0.0.1:33120_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:33120/collection1/
   [junit4]   2> 2723521 INFO  (coreZkRegister-5303-thread-1-processing-n:127.0.0.1:33120_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 2723521 INFO  (OverseerStateUpdate-96353883992948740-127.0.0.1:33120_-n_0000000000) [n:127.0.0.1:33120_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 2723522 INFO  (coreZkRegister-5303-thread-1-processing-n:127.0.0.1:33120_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:33120/collection1/ has no replicas
   [junit4]   2> 2723522 INFO  (coreZkRegister-5303-thread-1-processing-n:127.0.0.1:33120_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 2723524 INFO  (coreZkRegister-5303-thread-1-processing-n:127.0.0.1:33120_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/control_collection/leaders/shard1/leader after winning as /collections/control_collection/leader_elect/shard1/election/96353883992948740-core_node1-n_0000000000
   [junit4]   2> 2723526 INFO  (coreZkRegister-5303-thread-1-processing-n:127.0.0.1:33120_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:33120/collection1/ shard1
   [junit4]   2> 2723526 INFO  (OverseerStateUpdate-96353883992948740-127.0.0.1:33120_-n_0000000000) [n:127.0.0.1:33120_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:33120",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 1
   [junit4]   2> 2723628 INFO  (zkCallback-2778-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 2723677 INFO  (coreZkRegister-5303-thread-1-processing-n:127.0.0.1:33120_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:33120/collection1/ and leader is http://127.0.0.1:33120/collection1/
   [junit4]   2> 2723677 INFO  (coreZkRegister-5303-thread-1-processing-n:127.0.0.1:33120_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:33120
   [junit4]   2> 2723677 INFO  (coreZkRegister-5303-thread-1-processing-n:127.0.0.1:33120_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 2723677 INFO  (coreZkRegister-5303-thread-1-processing-n:127.0.0.1:33120_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 2723677 INFO  (coreZkRegister-5303-thread-1-processing-n:127.0.0.1:33120_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 2723678 INFO  (coreZkRegister-5303-thread-1-processing-n:127.0.0.1:33120_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Checking legacy cluster state for collection control_collection
   [junit4]   2> 2723678 INFO  (OverseerStateUpdate-96353883992948740-127.0.0.1:33120_-n_0000000000) [n:127.0.0.1:33120_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:33120",
   [junit4]   2>   "node_name":"127.0.0.1:33120_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"} current state version: 2
   [junit4]   2> 2723678 INFO  (OverseerStateUpdate-96353883992948740-127.0.0.1:33120_-n_0000000000) [n:127.0.0.1:33120_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:33120",
   [junit4]   2>   "node_name":"127.0.0.1:33120_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 2723780 INFO  (zkCallback-2778-thread-3-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 2723893 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2723893 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2723894 INFO  (zkCallback-2782-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@54e169f name:ZooKeeperConnection Watcher:127.0.0.1:50426/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2723894 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2723894 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2723894 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 2723894 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.c.ZkStateReader Loaded empty cluster properties
   [junit4]   2> 2723895 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 2723895 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 2723895 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with stateFormat=2
   [junit4]   2> 2723895 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2723896 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2723896 INFO  (zkCallback-2783-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@33aee513 name:ZooKeeperConnection Watcher:127.0.0.1:50426/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2723896 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2723896 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2723901 INFO  (OverseerStateUpdate-96353883992948740-127.0.0.1:33120_-n_0000000000) [n:127.0.0.1:33120_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"create",
   [junit4]   2>   "name":"collection1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "stateFormat":"2"} current state version: 3
   [junit4]   2> 2723901 INFO  (OverseerStateUpdate-96353883992948740-127.0.0.1:33120_-n_0000000000) [n:127.0.0.1:33120_    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 2723956 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_146D5B52C5AF51A-001/shard-1-001/cores/collection1
   [junit4]   2> 2723956 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_146D5B52C5AF51A-001/shard-1-001
   [junit4]   2> 2723957 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.e.j.s.Server jetty-9.3.8.v20160314
   [junit4]   2> 2723958 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@77da2d29{/,null,AVAILABLE}
   [junit4]   2> 2723958 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.e.j.s.ServerConnector Started ServerConnector@23f27303{HTTP/1.1,[http/1.1]}{127.0.0.1:46685}
   [junit4]   2> 2723959 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.e.j.s.Server Started @2726183ms
   [junit4]   2> 2723959 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_146D5B52C5AF51A-001/tempDir-001/jetty1, solrconfig=solrconfig.xml, hostContext=/, hostPort=46685, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_146D5B52C5AF51A-001/shard-1-001/cores}
   [junit4]   2> 2723959 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@73d16e93
   [junit4]   2> 2723959 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_146D5B52C5AF51A-001/shard-1-001'
   [junit4]   2> 2723959 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 2723959 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 2723959 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2723960 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2723960 INFO  (zkCallback-2784-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@320d42b2 name:ZooKeeperConnection Watcher:127.0.0.1:50426/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2723960 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2723961 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2723961 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 2723961 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_146D5B52C5AF51A-001/shard-1-001/solr.xml
   [junit4]   2> 2723964 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_146D5B52C5AF51A-001/shard-1-001/cores
   [junit4]   2> 2723964 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.CoreContainer New CoreContainer 1656449871
   [junit4]   2> 2723964 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_146D5B52C5AF51A-001/shard-1-001]
   [junit4]   2> 2723964 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_146D5B52C5AF51A-001/shard-1-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_146D5B52C5AF51A-001/shard-1-001/lib
   [junit4]   2> 2723965 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,connectionsEvictorSleepDelay : 5000,maxConnectionIdleTime : 40000,
   [junit4]   2> 2723965 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 2723966 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 2723966 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 2723966 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:50426/solr
   [junit4]   2> 2723966 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 2723966 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2723966 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2723967 INFO  (zkCallback-2787-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5504ee25 name:ZooKeeperConnection Watcher:127.0.0.1:50426 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2723968 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2723968 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2723968 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [n:127.0.0.1:46685_    ] o.a.s.c.ZkController Added new OnReconnect listener org.apache.solr.cloud.ZkController$$Lambda$12/976932739@2777a411
   [junit4]   2> 2723969 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [n:127.0.0.1:46685_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2723969 INFO  (zkCallback-2788-thread-1-processing-n:127.0.0.1:46685_) [n:127.0.0.1:46685_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@434ba52e name:ZooKeeperConnection Watcher:127.0.0.1:50426/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2723969 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [n:127.0.0.1:46685_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2723972 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [n:127.0.0.1:46685_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 2723972 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [n:127.0.0.1:46685_    ] o.a.s.c.c.ZkStateReader Loaded empty cluster properties
   [junit4]   2> 2723973 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [n:127.0.0.1:46685_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 2723974 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [n:127.0.0.1:46685_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 2723974 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [n:127.0.0.1:46685_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/96353883992948745-127.0.0.1:46685_-n_0000000001
   [junit4]   2> 2723974 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [n:127.0.0.1:46685_    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/96353883992948740-127.0.0.1:33120_-n_0000000000 to know if I could be the leader
   [junit4]   2> 2723975 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [n:127.0.0.1:46685_    ] o.a.s.c.ZkController Publish node=127.0.0.1:46685_ as DOWN
   [junit4]   2> 2723975 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [n:127.0.0.1:46685_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:46685_
   [junit4]   2> 2723975 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [n:127.0.0.1:46685_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:46685_
   [junit4]   2> 2723975 INFO  (OverseerStateUpdate-96353883992948740-127.0.0.1:33120_-n_0000000000) [n:127.0.0.1:33120_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"downnode",
   [junit4]   2>   "node_name":"127.0.0.1:46685_"} current state version: 3
   [junit4]   2> 2723975 INFO  (OverseerStateUpdate-96353883992948740-127.0.0.1:33120_-n_0000000000) [n:127.0.0.1:33120_    ] o.a.s.c.o.NodeMutator DownNode state invoked for node: 127.0.0.1:46685_
   [junit4]   2> 2723976 INFO  (OverseerStateUpdate-96353883992948740-127.0.0.1:33120_-n_0000000000) [n:127.0.0.1:33120_    ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/collection1/state.json
   [junit4]   2> 2723976 INFO  (zkCallback-2782-thread-1) [    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 2723976 INFO  (zkCallback-2778-thread-2-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 2723976 INFO  (zkCallback-2782-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 2723976 INFO  (zkCallback-2788-thread-1-processing-n:127.0.0.1:46685_) [n:127.0.0.1:46685_    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 2723976 INFO  (zkCallback-2778-thread-2-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 2723976 INFO  (zkCallback-2788-thread-1-processing-n:127.0.0.1:46685_) [n:127.0.0.1:46685_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 2723977 INFO  (zkCallback-2778-thread-2-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2723977 INFO  (zkCallback-2782-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2723977 INFO  (zkCallback-2788-thread-1-processing-n:127.0.0.1:46685_) [n:127.0.0.1:46685_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2723982 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [n:127.0.0.1:46685_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 2723982 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [n:127.0.0.1:46685_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 2723983 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [n:127.0.0.1:46685_    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_146D5B52C5AF51A-001/shard-1-001/cores
   [junit4]   2> 2723983 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [n:127.0.0.1:46685_    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, loadOnStartup=true, configSetProperties=configsetprops.json, collection=collection1, coreNodeName=, dataDir=data/, shard=}
   [junit4]   2> 2723983 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [n:127.0.0.1:46685_    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_146D5B52C5AF51A-001/shard-1-001/cores/collection1
   [junit4]   2> 2723983 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [n:127.0.0.1:46685_    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 2723984 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [n:127.0.0.1:46685_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2
   [junit4]   2> 2723984 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [n:127.0.0.1:46685_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 2723984 INFO  (coreLoadExecutor-5320-thread-1-processing-n:127.0.0.1:46685_) [n:127.0.0.1:46685_ c:collection1   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 2723984 INFO  (coreLoadExecutor-5320-thread-1-processing-n:127.0.0.1:46685_) [n:127.0.0.1:46685_ c:collection1   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 2723985 INFO  (coreLoadExecutor-5320-thread-1-processing-n:127.0.0.1:46685_) [n:127.0.0.1:46685_ c:collection1   x:collection1] o.a.s.c.ZkController Registering watch for collection collection1
   [junit4]   2> 2723985 INFO  (OverseerStateUpdate-96353883992948740-127.0.0.1:33120_-n_0000000000) [n:127.0.0.1:33120_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:46685",
   [junit4]   2>   "node_name":"127.0.0.1:46685_",
   [junit4]   2>   

[...truncated too long message...]

exception-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-journal-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-normalization-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-operational-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-referral-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-schema-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-subtree-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-trigger-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-jdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-jdbm1-2.0.0-M2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-kerberos-codec-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-ldif-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-mavibot-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-protocol-kerberos-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-protocol-ldap-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-protocol-shared-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-xdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/api-all-1.0.0-M20.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/bcprov-jdk15-1.45.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/cglib-nodep-2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/commons-collections-3.2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/commons-math3-3.4.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/easymock-3.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/ehcache-core-2.4.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/hadoop-common-2.7.2-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/hadoop-hdfs-2.7.2-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/hadoop-minikdc-2.7.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/htrace-core-3.2.0-incubating.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jackson-annotations-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jackson-core-asl-1.9.13.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jackson-databind-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jackson-mapper-asl-1.9.13.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jersey-core-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jersey-server-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jetty-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jetty-util-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/mina-core-2.0.0-M5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/mockito-core-1.9.5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/netty-all-4.0.36.Final.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/objenesis-1.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/presto-parser-0.122.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/slice-0.10.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/analysis/icu/lucene-analyzers-icu-6.2.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/contrib/solr-analysis-extras/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/queryparser/classes/test:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/contrib/analysis-extras/lib/icu4j-56.1.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-launcher.jar:/x1/jenkins/.ant/lib/ivy-2.3.0.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-antlr.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-regexp.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-resolver.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-net.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jmf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-oro.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-xalan2.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-logging.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jsch.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit4.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-javamail.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jai.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jdepend.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bsf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-swing.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-netrexx.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bcel.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-testutil.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-log4j.jar:/x1/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.8/lib/tools.jar:/x1/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.3.4.jar com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/temp/junit4-J1-20160803_161425_766.events @/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/temp/junit4-J1-20160803_161425_766.suites -stdin
   [junit4] ERROR: JVM J1 ended with an exception: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1501)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:121)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:955)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:952)
   [junit4] 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4] 	at java.lang.Thread.run(Thread.java:745)

BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:771: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:707: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/common-build.xml:536: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/common-build.xml:1443: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/common-build.xml:984: At least one slave process threw an exception, first: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.

Total time: 264 minutes 48 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any




[JENKINS] Lucene-Solr-NightlyTests-6.x - Build # 134 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-6.x/134/

3 tests failed.
FAILED:  org.apache.solr.cloud.TestRandomRequestDistribution.test

Error Message:
Shard a1x2_shard1_replica1 received all 10 requests

Stack Trace:
java.lang.AssertionError: Shard a1x2_shard1_replica1 received all 10 requests
	at __randomizedtesting.SeedInfo.seed([146FB096C07DA498:9C3B8F4C6E81C960]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.apache.solr.cloud.TestRandomRequestDistribution.testRequestTracking(TestRandomRequestDistribution.java:121)
	at org.apache.solr.cloud.TestRandomRequestDistribution.test(TestRandomRequestDistribution.java:65)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  junit.framework.TestSuite.org.apache.solr.handler.TestReplicationHandler

Error Message:
ObjectTracker found 10 object(s) that were not released!!! [NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory]

Stack Trace:
java.lang.AssertionError: ObjectTracker found 10 object(s) that were not released!!! [NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory]
	at __randomizedtesting.SeedInfo.seed([146FB096C07DA498]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.junit.Assert.assertNull(Assert.java:551)
	at org.apache.solr.SolrTestCaseJ4.teardownTestCases(SolrTestCaseJ4.java:258)
	at sun.reflect.GeneratedMethodAccessor23.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:834)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.core.TestDynamicLoading.testDynamicLoading

Error Message:
Could not get expected value  'X val' for path 'x' full output: {   "responseHeader":{     "status":0,     "QTime":0},   "params":{"wt":"json"},   "context":{     "webapp":"/xgjyk",     "path":"/test1",     "httpMethod":"GET"},   "class":"org.apache.solr.core.BlobStoreTestRequestHandler",   "x":null},  from server:  null

Stack Trace:
java.lang.AssertionError: Could not get expected value  'X val' for path 'x' full output: {
  "responseHeader":{
    "status":0,
    "QTime":0},
  "params":{"wt":"json"},
  "context":{
    "webapp":"/xgjyk",
    "path":"/test1",
    "httpMethod":"GET"},
  "class":"org.apache.solr.core.BlobStoreTestRequestHandler",
  "x":null},  from server:  null
	at __randomizedtesting.SeedInfo.seed([146FB096C07DA498:CC229DC137A00138]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.apache.solr.core.TestSolrConfigHandler.testForResponseElement(TestSolrConfigHandler.java:481)
	at org.apache.solr.core.TestDynamicLoading.testDynamicLoading(TestDynamicLoading.java:232)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 11981 lines...]
   [junit4] Suite: org.apache.solr.handler.TestReplicationHandler
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/init-core-data-001
   [junit4]   2> 2025312 INFO  (SUITE-TestReplicationHandler-seed#[146FB096C07DA498]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=None)
   [junit4]   2> 2025313 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.SolrTestCaseJ4 ###Starting doTestReplicateAfterStartupWithNoActivity
   [junit4]   2> 2025314 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001/collection1
   [junit4]   2> 2025322 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.e.j.s.Server jetty-9.3.8.v20160314
   [junit4]   2> 2025323 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@2b35cd35{/solr,null,AVAILABLE}
   [junit4]   2> 2025325 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.e.j.s.ServerConnector Started ServerConnector@1c6edbed{HTTP/1.1,[http/1.1]}{127.0.0.1:43550}
   [junit4]   2> 2025325 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.e.j.s.Server Started @2027626ms
   [junit4]   2> 2025325 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001/collection1/data, hostContext=/solr, hostPort=43550}
   [junit4]   2> 2025325 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@73d16e93
   [junit4]   2> 2025325 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001'
   [junit4]   2> 2025325 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 2025325 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 2025325 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001/solr.xml
   [junit4]   2> 2025329 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001/.
   [junit4]   2> 2025329 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.c.CoreContainer New CoreContainer 2059679889
   [junit4]   2> 2025329 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001]
   [junit4]   2> 2025329 WARN  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001/lib
   [junit4]   2> 2025329 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,connectionsEvictorSleepDelay : 5000,maxConnectionIdleTime : 40000,
   [junit4]   2> 2025330 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 2025330 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 2025330 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 2025331 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 2025331 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 2025331 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001/.
   [junit4]   2> 2025332 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {name=collection1, config=solrconfig.xml, loadOnStartup=true, schema=schema.xml, configSetProperties=configsetprops.json, transient=false, dataDir=data/}
   [junit4]   2> 2025332 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001/./collection1
   [junit4]   2> 2025332 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 2025332 INFO  (coreLoadExecutor-4278-thread-1) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001/collection1'
   [junit4]   2> 2025332 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0
   [junit4]   2> 2025333 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 2025333 INFO  (coreLoadExecutor-4278-thread-1) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 2025333 INFO  (coreLoadExecutor-4278-thread-1) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 2025339 INFO  (coreLoadExecutor-4278-thread-1) [    ] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 2025341 WARN  (coreLoadExecutor-4278-thread-1) [    ] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
   [junit4]   2> 2025342 INFO  (coreLoadExecutor-4278-thread-1) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.2.0
   [junit4]   2> 2025346 INFO  (coreLoadExecutor-4278-thread-1) [    ] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 2025348 INFO  (coreLoadExecutor-4278-thread-1) [    ] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 2025350 INFO  (coreLoadExecutor-4278-thread-1) [    ] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 2025351 INFO  (coreLoadExecutor-4278-thread-1) [    ] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001/collection1'
   [junit4]   2> 2025351 INFO  (coreLoadExecutor-4278-thread-1) [    ] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001/./collection1
   [junit4]   2> 2025351 INFO  (coreLoadExecutor-4278-thread-1) [    x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 2025351 INFO  (coreLoadExecutor-4278-thread-1) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001/./collection1/data/]
   [junit4]   2> 2025351 INFO  (coreLoadExecutor-4278-thread-1) [    x:collection1] o.a.s.c.SolrCore JMX monitoring not detected for core: collection1
   [junit4]   2> 2025352 INFO  (coreLoadExecutor-4278-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001/./collection1/data/snapshot_metadata
   [junit4]   2> 2025352 INFO  (coreLoadExecutor-4278-thread-1) [    x:collection1] o.a.s.c.s.SolrSnapshotMetaDataManager Loading from snapshot metadata file...
   [junit4]   2> 2025352 INFO  (coreLoadExecutor-4278-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001/./collection1/data
   [junit4]   2> 2025352 INFO  (coreLoadExecutor-4278-thread-1) [    x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001/./collection1/data/index/
   [junit4]   2> 2025352 WARN  (coreLoadExecutor-4278-thread-1) [    x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001/./collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 2025352 INFO  (coreLoadExecutor-4278-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001/./collection1/data/index
   [junit4]   2> 2025352 INFO  (coreLoadExecutor-4278-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=50, maxMergeAtOnceExplicit=40, maxMergedSegmentMB=65.857421875, floorSegmentMB=2.123046875, forceMergeDeletesPctAllowed=10.510085272075946, segmentsPerTier=46.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.706243987464695
   [junit4]   2> 2025352 INFO  (coreLoadExecutor-4278-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@4defe29e lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@521f464f),segFN=segments_1,generation=1}
   [junit4]   2> 2025353 INFO  (coreLoadExecutor-4278-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 2025353 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [    x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001/./collection1/data/
   [junit4]   2> 2025354 INFO  (coreLoadExecutor-4278-thread-1) [    x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 2025355 INFO  (coreLoadExecutor-4278-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2025355 INFO  (coreLoadExecutor-4278-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2025356 INFO  (coreLoadExecutor-4278-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2025357 INFO  (coreLoadExecutor-4278-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2025357 INFO  (coreLoadExecutor-4278-thread-1) [    x:collection1] o.a.s.c.PluginBag requestHandler : 'lazy' created with startup=lazy 
   [junit4]   2> 2025357 INFO  (coreLoadExecutor-4278-thread-1) [    x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/mbeans,standard,lazy,/update/json/docs,/admin/luke,/export,/get,/admin/properties,/update/json,/admin/threads,/update/csv,/sql,/graph,/admin/segments,/admin/system,/replication,defaults,/config,/stream,/schema,/admin/plugins,/admin/logging,/admin/ping,/update,/admin/file,/terms
   [junit4]   2> 2025357 INFO  (coreLoadExecutor-4278-thread-1) [    x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 2025358 INFO  (coreLoadExecutor-4278-thread-1) [    x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 2025358 INFO  (coreLoadExecutor-4278-thread-1) [    x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 2025358 INFO  (coreLoadExecutor-4278-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=211309566, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
   [junit4]   2> 2025358 INFO  (coreLoadExecutor-4278-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@4defe29e lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@521f464f),segFN=segments_1,generation=1}
   [junit4]   2> 2025358 INFO  (coreLoadExecutor-4278-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 2025358 INFO  (coreLoadExecutor-4278-thread-1) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@6b8ecd57[collection1] main]
   [junit4]   2> 2025358 INFO  (coreLoadExecutor-4278-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001/collection1/conf
   [junit4]   2> 2025358 INFO  (coreLoadExecutor-4278-thread-1) [    x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {storageDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001/collection1/conf}
   [junit4]   2> 2025358 INFO  (coreLoadExecutor-4278-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using file:dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001/collection1/conf
   [junit4]   2> 2025358 INFO  (coreLoadExecutor-4278-thread-1) [    x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 2025359 INFO  (coreLoadExecutor-4278-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler Replication enabled for following config files: schema.xml,xslt/dummy.xsl
   [junit4]   2> 2025359 INFO  (coreLoadExecutor-4278-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 2025359 INFO  (searcherExecutor-4279-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@6b8ecd57[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2025359 INFO  (coreLoadExecutor-4278-thread-1) [    x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 2025834 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-002/collection1
   [junit4]   2> 2025838 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.e.j.s.Server jetty-9.3.8.v20160314
   [junit4]   2> 2025839 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@37f99303{/solr,null,AVAILABLE}
   [junit4]   2> 2025839 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.e.j.s.ServerConnector Started ServerConnector@2313dc1c{HTTP/1.1,[http/1.1]}{127.0.0.1:34886}
   [junit4]   2> 2025839 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.e.j.s.Server Started @2028141ms
   [junit4]   2> 2025840 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-002/collection1/data, hostContext=/solr, hostPort=34886}
   [junit4]   2> 2025840 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@73d16e93
   [junit4]   2> 2025840 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-002'
   [junit4]   2> 2025842 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 2025842 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 2025844 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-002/solr.xml
   [junit4]   2> 2025847 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-002/.
   [junit4]   2> 2025848 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.c.CoreContainer New CoreContainer 502734745
   [junit4]   2> 2025848 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-002]
   [junit4]   2> 2025848 WARN  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-002/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-002/lib
   [junit4]   2> 2025848 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,connectionsEvictorSleepDelay : 5000,maxConnectionIdleTime : 40000,
   [junit4]   2> 2025849 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 2025849 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 2025849 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 2025849 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 2025849 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 2025850 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-002/.
   [junit4]   2> 2025850 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {name=collection1, config=solrconfig.xml, loadOnStartup=true, schema=schema.xml, configSetProperties=configsetprops.json, transient=false, dataDir=data/}
   [junit4]   2> 2025850 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-002/./collection1
   [junit4]   2> 2025850 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 2025853 INFO  (coreLoadExecutor-4288-thread-1) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-002/collection1'
   [junit4]   2> 2025855 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0
   [junit4]   2> 2025855 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 2025855 INFO  (coreLoadExecutor-4288-thread-1) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 2025855 INFO  (coreLoadExecutor-4288-thread-1) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 2025858 INFO  (coreLoadExecutor-4288-thread-1) [    ] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 2025860 WARN  (coreLoadExecutor-4288-thread-1) [    ] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
   [junit4]   2> 2025861 INFO  (coreLoadExecutor-4288-thread-1) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.2.0
   [junit4]   2> 2025865 INFO  (coreLoadExecutor-4288-thread-1) [    ] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 2025867 INFO  (coreLoadExecutor-4288-thread-1) [    ] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 2025868 INFO  (coreLoadExecutor-4288-thread-1) [    ] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 2025869 INFO  (coreLoadExecutor-4288-thread-1) [    ] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-002/collection1'
   [junit4]   2> 2025869 INFO  (coreLoadExecutor-4288-thread-1) [    ] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-002/./collection1
   [junit4]   2> 2025869 INFO  (coreLoadExecutor-4288-thread-1) [    x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 2025869 INFO  (coreLoadExecutor-4288-thread-1) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-002/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-002/./collection1/data/]
   [junit4]   2> 2025869 INFO  (coreLoadExecutor-4288-thread-1) [    x:collection1] o.a.s.c.SolrCore JMX monitoring not detected for core: collection1
   [junit4]   2> 2025870 INFO  (coreLoadExecutor-4288-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-002/./collection1/data/snapshot_metadata
   [junit4]   2> 2025870 INFO  (coreLoadExecutor-4288-thread-1) [    x:collection1] o.a.s.c.s.SolrSnapshotMetaDataManager Loading from snapshot metadata file...
   [junit4]   2> 2025870 INFO  (coreLoadExecutor-4288-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-002/./collection1/data
   [junit4]   2> 2025870 INFO  (coreLoadExecutor-4288-thread-1) [    x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-002/./collection1/data/index/
   [junit4]   2> 2025870 WARN  (coreLoadExecutor-4288-thread-1) [    x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-002/./collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 2025870 INFO  (coreLoadExecutor-4288-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-002/./collection1/data/index
   [junit4]   2> 2025870 INFO  (coreLoadExecutor-4288-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=50, maxMergeAtOnceExplicit=40, maxMergedSegmentMB=65.857421875, floorSegmentMB=2.123046875, forceMergeDeletesPctAllowed=10.510085272075946, segmentsPerTier=46.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.706243987464695
   [junit4]   2> 2025870 INFO  (coreLoadExecutor-4288-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@c7fcbce lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@29feffae),segFN=segments_1,generation=1}
   [junit4]   2> 2025870 INFO  (coreLoadExecutor-4288-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 2025871 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [    x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-002/./collection1/data/
   [junit4]   2> 2025872 INFO  (coreLoadExecutor-4288-thread-1) [    x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 2025872 INFO  (coreLoadExecutor-4288-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2025873 INFO  (coreLoadExecutor-4288-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2025873 INFO  (coreLoadExecutor-4288-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2025874 INFO  (coreLoadExecutor-4288-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2025874 INFO  (coreLoadExecutor-4288-thread-1) [    x:collection1] o.a.s.c.PluginBag requestHandler : 'lazy' created with startup=lazy 
   [junit4]   2> 2025875 INFO  (coreLoadExecutor-4288-thread-1) [    x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/mbeans,standard,lazy,/update/json/docs,/admin/luke,/export,/get,/admin/properties,/update/json,/admin/threads,/update/csv,/sql,/graph,/admin/segments,/admin/system,/replication,defaults,/config,/stream,/schema,/admin/plugins,/admin/logging,/admin/ping,/update,/admin/file,/terms
   [junit4]   2> 2025875 INFO  (coreLoadExecutor-4288-thread-1) [    x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 2025875 INFO  (coreLoadExecutor-4288-thread-1) [    x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 2025875 INFO  (coreLoadExecutor-4288-thread-1) [    x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 2025875 INFO  (coreLoadExecutor-4288-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=211309566, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
   [junit4]   2> 2025875 INFO  (coreLoadExecutor-4288-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@c7fcbce lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@29feffae),segFN=segments_1,generation=1}
   [junit4]   2> 2025875 INFO  (coreLoadExecutor-4288-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 2025876 INFO  (coreLoadExecutor-4288-thread-1) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@386f0884[collection1] main]
   [junit4]   2> 2025876 INFO  (coreLoadExecutor-4288-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-002/collection1/conf
   [junit4]   2> 2025876 INFO  (coreLoadExecutor-4288-thread-1) [    x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {storageDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-002/collection1/conf}
   [junit4]   2> 2025876 INFO  (coreLoadExecutor-4288-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using file:dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-002/collection1/conf
   [junit4]   2> 2025876 INFO  (coreLoadExecutor-4288-thread-1) [    x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 2025877 INFO  (coreLoadExecutor-4288-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler Poll scheduled at an interval of 1000ms
   [junit4]   2> 2025877 INFO  (searcherExecutor-4289-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@386f0884[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2025877 INFO  (coreLoadExecutor-4288-thread-1) [    x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 2026356 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.e.j.s.ServerConnector Stopped ServerConnector@2313dc1c{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
   [junit4]   2> 2026356 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=502734745
   [junit4]   2> 2026357 INFO  (coreCloseExecutor-4294-thread-1) [    x:collection1] o.a.s.c.SolrCore [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@388fe234
   [junit4]   2> 2026357 INFO  (coreCloseExecutor-4294-thread-1) [    x:collection1] o.a.s.u.DirectUpdateHandler2 closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
   [junit4]   2> 2026357 INFO  (coreCloseExecutor-4294-thread-1) [    x:collection1] o.a.s.u.SolrCoreState Closing SolrCoreState
   [junit4]   2> 2026357 INFO  (coreCloseExecutor-4294-thread-1) [    x:collection1] o.a.s.u.DefaultSolrCoreState SolrCoreState ref count has reached 0 - closing IndexWriter
   [junit4]   2> 2026357 INFO  (coreCloseExecutor-4294-thread-1) [    x:collection1] o.a.s.u.DefaultSolrCoreState closing IndexWriter with IndexWriterCloser
   [junit4]   2> 2026357 INFO  (coreCloseExecutor-4294-thread-1) [    x:collection1] o.a.s.c.SolrCore [collection1] Closing main searcher on request.
   [junit4]   2> 2026357 INFO  (coreCloseExecutor-4294-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing MockDirectoryFactory - 3 directories currently being tracked
   [junit4]   2> 2026357 INFO  (coreCloseExecutor-4294-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory looking to close /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-002/./collection1/data [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-002/./collection1/data;done=false>>]
   [junit4]   2> 2026357 INFO  (coreCloseExecutor-4294-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory, CoreContainer#isShutdown=true
   [junit4]   2> 2026357 INFO  (coreCloseExecutor-4294-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-002/./collection1/data
   [junit4]   2> 2026357 INFO  (coreCloseExecutor-4294-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory looking to close /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-002/./collection1/data/snapshot_metadata [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-002/./collection1/data/snapshot_metadata;done=false>>]
   [junit4]   2> 2026357 INFO  (coreCloseExecutor-4294-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory, CoreContainer#isShutdown=true
   [junit4]   2> 2026357 INFO  (coreCloseExecutor-4294-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-002/./collection1/data/snapshot_metadata
   [junit4]   2> 2026357 INFO  (coreCloseExecutor-4294-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory looking to close /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-002/./collection1/data/index [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-002/./collection1/data/index;done=false>>]
   [junit4]   2> 2026358 INFO  (coreCloseExecutor-4294-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory, CoreContainer#isShutdown=true
   [junit4]   2> 2026358 INFO  (coreCloseExecutor-4294-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-002/./collection1/data/index
   [junit4]   2> 2026358 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@37f99303{/solr,null,UNAVAILABLE}
   [junit4]   2> 2026361 INFO  (qtp221511252-59611) [    x:collection1] o.a.s.u.DirectUpdateHandler2 [collection1] REMOVING ALL DOCUMENTS FROM INDEX
   [junit4]   2> 2026361 INFO  (qtp221511252-59611) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{deleteByQuery=*:*} 0 0
   [junit4]   2> 2026363 INFO  (qtp221511252-59612) [    x:collection1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 2026364 INFO  (qtp221511252-59612) [    x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@4defe29e lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@521f464f),segFN=segments_1,generation=1}
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@4defe29e lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@521f464f),segFN=segments_2,generation=2}
   [junit4]   2> 2026365 INFO  (qtp221511252-59612) [    x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 2
   [junit4]   2> 2026367 INFO  (qtp221511252-59612) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@34acf75b[collection1] main]
   [junit4]   2> 2026367 INFO  (qtp221511252-59612) [    x:collection1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 2026367 INFO  (searcherExecutor-4279-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@34acf75b[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2026367 INFO  (qtp221511252-59612) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2}{commit=} 0 4
   [junit4]   2> 2026369 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.e.j.s.ServerConnector Stopped ServerConnector@1c6edbed{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
   [junit4]   2> 2026369 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=2059679889
   [junit4]   2> 2026371 INFO  (coreCloseExecutor-4296-thread-1) [    x:collection1] o.a.s.c.SolrCore [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@733c287b
   [junit4]   2> 2026371 INFO  (coreCloseExecutor-4296-thread-1) [    x:collection1] o.a.s.u.DirectUpdateHandler2 closing DirectUpdateHandler2{commits=1,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=1,cumulative_errors=0}
   [junit4]   2> 2026372 INFO  (coreCloseExecutor-4296-thread-1) [    x:collection1] o.a.s.u.SolrCoreState Closing SolrCoreState
   [junit4]   2> 2026372 INFO  (coreCloseExecutor-4296-thread-1) [    x:collection1] o.a.s.u.DefaultSolrCoreState SolrCoreState ref count has reached 0 - closing IndexWriter
   [junit4]   2> 2026372 INFO  (coreCloseExecutor-4296-thread-1) [    x:collection1] o.a.s.u.DefaultSolrCoreState closing IndexWriter with IndexWriterCloser
   [junit4]   2> 2026372 INFO  (coreCloseExecutor-4296-thread-1) [    x:collection1] o.a.s.c.SolrCore [collection1] Closing main searcher on request.
   [junit4]   2> 2026372 INFO  (coreCloseExecutor-4296-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing MockDirectoryFactory - 3 directories currently being tracked
   [junit4]   2> 2026372 INFO  (coreCloseExecutor-4296-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory looking to close /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001/./collection1/data [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001/./collection1/data;done=false>>]
   [junit4]   2> 2026372 INFO  (coreCloseExecutor-4296-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory, CoreContainer#isShutdown=true
   [junit4]   2> 2026372 INFO  (coreCloseExecutor-4296-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001/./collection1/data
   [junit4]   2> 2026372 INFO  (coreCloseExecutor-4296-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory looking to close /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001/./collection1/data/index [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001/./collection1/data/index;done=false>>]
   [junit4]   2> 2026372 INFO  (coreCloseExecutor-4296-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory, CoreContainer#isShutdown=true
   [junit4]   2> 2026372 INFO  (coreCloseExecutor-4296-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001/./collection1/data/index
   [junit4]   2> 2026372 INFO  (coreCloseExecutor-4296-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory looking to close /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001/./collection1/data/snapshot_metadata [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001/./collection1/data/snapshot_metadata;done=false>>]
   [junit4]   2> 2026372 INFO  (coreCloseExecutor-4296-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory, CoreContainer#isShutdown=true
   [junit4]   2> 2026372 INFO  (coreCloseExecutor-4296-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001/./collection1/data/snapshot_metadata
   [junit4]   2> 2026373 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@2b35cd35{/solr,null,UNAVAILABLE}
   [junit4]   2> 2026374 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.e.j.s.Server jetty-9.3.8.v20160314
   [junit4]   2> 2026378 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@9985892{/solr,null,AVAILABLE}
   [junit4]   2> 2026378 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.e.j.s.ServerConnector Started ServerConnector@42be38f8{HTTP/1.1,[http/1.1]}{127.0.0.1:57422}
   [junit4]   2> 2026378 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.e.j.s.Server Started @2028680ms
   [junit4]   2> 2026378 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001/collection1/data, hostContext=/solr, hostPort=57422}
   [junit4]   2> 2026379 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@73d16e93
   [junit4]   2> 2026379 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001'
   [junit4]   2> 2026379 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 2026379 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 2026379 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001/solr.xml
   [junit4]   2> 2026382 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001/.
   [junit4]   2> 2026382 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.c.CoreContainer New CoreContainer 1145471155
   [junit4]   2> 2026382 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001]
   [junit4]   2> 2026382 WARN  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001/lib
   [junit4]   2> 2026383 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,connectionsEvictorSleepDelay : 5000,maxConnectionIdleTime : 40000,
   [junit4]   2> 2026386 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 2026386 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 2026386 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 2026386 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 2026386 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 2026386 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001/.
   [junit4]   2> 2026387 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {name=collection1, config=solrconfig.xml, loadOnStartup=true, schema=schema.xml, configSetProperties=configsetprops.json, transient=false, dataDir=data/}
   [junit4]   2> 2026387 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001/./collection1
   [junit4]   2> 2026387 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 2026387 INFO  (coreLoadExecutor-4303-thread-1) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001/collection1'
   [junit4]   2> 2026395 INFO  (coreLoadExecutor-4303-thread-1) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 2026395 INFO  (coreLoadExecutor-4303-thread-1) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 2026395 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0
   [junit4]   2> 2026395 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 2026398 INFO  (coreLoadExecutor-4303-thread-1) [    ] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 2026400 WARN  (coreLoadExecutor-4303-thread-1) [    ] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
   [junit4]   2> 2026400 INFO  (coreLoadExecutor-4303-thread-1) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.2.0
   [junit4]   2> 2026405 INFO  (coreLoadExecutor-4303-thread-1) [    ] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 2026406 INFO  (coreLoadExecutor-4303-thread-1) [    ] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 2026408 INFO  (coreLoadExecutor-4303-thread-1) [    ] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 2026408 INFO  (coreLoadExecutor-4303-thread-1) [    ] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001/collection1'
   [junit4]   2> 2026408 INFO  (coreLoadExecutor-4303-thread-1) [    ] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001/./collection1
   [junit4]   2> 2026409 INFO  (coreLoadExecutor-4303-thread-1) [    x:collection1] o.a.s.c.SolrCore solr.NRTCachingDirectoryFactory
   [junit4]   2> 2026413 INFO  (coreLoadExecutor-4303-thread-1) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001/./collection1/data/]
   [junit4]   2> 2026413 INFO  (coreLoadExecutor-4303-thread-1) [    x:collection1] o.a.s.c.SolrCore JMX monitoring not detected for core: collection1
   [junit4]   2> 2026413 INFO  (coreLoadExecutor-4303-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001/collection1/data/snapshot_metadata
   [junit4]   2> 2026413 INFO  (coreLoadExecutor-4303-thread-1) [    x:collection1] o.a.s.c.s.SolrSnapshotMetaDataManager Loading from snapshot metadata file...
   [junit4]   2> 2026413 INFO  (coreLoadExecutor-4303-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001/collection1/data
   [junit4]   2> 2026413 INFO  (coreLoadExecutor-4303-thread-1) [    x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001/./collection1/data/index/
   [junit4]   2> 2026413 WARN  (coreLoadExecutor-4303-thread-1) [    x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001/./collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 2026414 INFO  (coreLoadExecutor-4303-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001/collection1/data/index
   [junit4]   2> 2026414 INFO  (coreLoadExecutor-4303-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=5, maxMergeAtOnceExplicit=9, maxMergedSegmentMB=39.6455078125, floorSegmentMB=1.498046875, forceMergeDeletesPctAllowed=5.661925787057958, segmentsPerTier=50.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.4052861862136866
   [junit4]   2> 2026418 INFO  (coreLoadExecutor-4303-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(MMapDirectory@/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001/collection1/data/index lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@688343a8; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
   [junit4]   2> 2026418 INFO  (coreLoadExecutor-4303-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 2026418 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [    x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001/./collection1/data/
   [junit4]   2> 2026419 INFO  (coreLoadExecutor-4303-thread-1) [    x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 2026420 INFO  (coreLoadExecutor-4303-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2026420 INFO  (coreLoadExecutor-4303-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2026421 INFO  (coreLoadExecutor-4303-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2026421 INFO  (coreLoadExecutor-4303-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2026422 INFO  (coreLoadExecutor-4303-thread-1) [    x:collection1] o.a.s.c.PluginBag requestHandler : 'lazy' created with startup=lazy 
   [junit4]   2> 2026422 INFO  (coreLoadExecutor-4303-thread-1) [    x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/mbeans,standard,lazy,/update/json/docs,/admin/luke,/export,/get,/admin/properties,/update/json,/admin/threads,/update/csv,/sql,/graph,/admin/segments,/admin/system,/replication,defaults,/config,/stream,/schema,/admin/plugins,/admin/logging,/admin/ping,/update,/admin/file,/terms
   [junit4]   2> 2026422 INFO  (coreLoadExecutor-4303-thread-1) [    x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 2026422 INFO  (coreLoadExecutor-4303-thread-1) [    x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 2026422 INFO  (coreLoadExecutor-4303-thread-1) [    x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 2026422 INFO  (coreLoadExecutor-4303-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=40, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.2181944154849793]
   [junit4]   2> 2026423 INFO  (coreLoadExecutor-4303-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(MMapDirectory@/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001/collection1/data/index lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@688343a8; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
   [junit4]   2> 2026423 INFO  (coreLoadExecutor-4303-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 2026423 INFO  (coreLoadExecutor-4303-thread-1) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@30a979dd[collection1] main]
   [junit4]   2> 2026423 INFO  (coreLoadExecutor-4303-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001/collection1/conf
   [junit4]   2> 2026423 INFO  (coreLoadExecutor-4303-thread-1) [    x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {storageDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001/collection1/conf}
   [junit4]   2> 2026423 INFO  (coreLoadExecutor-4303-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using file:dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001/collection1/conf
   [junit4]   2> 2026423 INFO  (coreLoadExecutor-4303-thread-1) [    x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 2026423 INFO  (coreLoadExecutor-4303-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler Replication enabled for following config files: schema.xml
   [junit4]   2> 2026423 INFO  (coreLoadExecutor-4303-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 2026423 INFO  (searcherExecutor-4304-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@30a979dd[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2026423 INFO  (coreLoadExecutor-4303-thread-1) [    x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 2026898 INFO  (qtp1268612900-59653) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[0]} 0 0
   [junit4]   2> 2026900 INFO  (qtp1268612900-59654) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[1]} 0 0
   [junit4]   2> 2026902 INFO  (qtp1268612900-59655) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[2]} 0 0
   [junit4]   2> 2026904 INFO  (qtp1268612900-59656) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[3]} 0 0
   [junit4]   2> 2026907 INFO  (qtp1268612900-59657) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[4]} 0 0
   [junit4]   2> 2026909 INFO  (qtp1268612900-59650) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[5]} 0 0
   [junit4]   2> 2026910 INFO  (qtp1268612900-59650) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[6]} 0 0
   [junit4]   2> 2026912 INFO  (qtp1268612900-59654) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[7]} 0 0
   [junit4]   2> 2026914 INFO  (qtp1268612900-59655) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[8]} 0 0
   [junit4]   2> 2026915 INFO  (qtp1268612900-59655) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[9]} 0 0
   [junit4]   2> 2026920 INFO  (qtp1268612900-59657) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[10]} 0 0
   [junit4]   2> 2026922 INFO  (qtp1268612900-59653) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[11]} 0 0
   [junit4]   2> 2026923 INFO  (qtp1268612900-59650) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[12]} 0 0
   [junit4]   2> 2026925 INFO  (qtp1268612900-59650) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[13]} 0 0
   [junit4]   2> 2026927 INFO  (qtp1268612900-59656) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[14]} 0 0
   [junit4]   2> 2026928 INFO  (qtp1268612900-59655) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[15]} 0 0
   [junit4]   2> 2026930 INFO  (qtp1268612900-59657) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[16]} 0 0
   [junit4]   2> 2026932 INFO  (qtp1268612900-59653) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[17]} 0 0
   [junit4]   2> 2026933 INFO  (qtp1268612900-59654) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[18]} 0 0
   [junit4]   2> 2026935 INFO  (qtp1268612900-59650) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[19]} 0 0
   [junit4]   2> 2026936 INFO  (qtp1268612900-59656) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[20]} 0 0
   [junit4]   2> 2026938 INFO  (qtp1268612900-59656) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[21]} 0 0
   [junit4]   2> 2026940 INFO  (qtp1268612900-59656) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[22]} 0 0
   [junit4]   2> 2026941 INFO  (qtp1268612900-59653) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[23]} 0 0
   [junit4]   2> 2026943 INFO  (qtp1268612900-59654) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[24]} 0 0
   [junit4]   2> 2026944 INFO  (qtp1268612900-59650) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[25]} 0 0
   [junit4]   2> 2026946 INFO  (qtp1268612900-59655) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[26]} 0 0
   [junit4]   2> 2026947 INFO  (qtp1268612900-59657) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[27]} 0 0
   [junit4]   2> 2026949 INFO  (qtp1268612900-59656) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[28]} 0 0
   [junit4]   2> 2026951 INFO  (qtp1268612900-59653) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[29]} 0 0
   [junit4]   2> 2026953 INFO  (qtp1268612900-59654) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[30]} 0 0
   [junit4]   2> 2026955 INFO  (qtp1268612900-59650) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[31]} 0 0
   [junit4]   2> 2026956 INFO  (qtp1268612900-59655) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[32]} 0 0
   [junit4]   2> 2026958 INFO  (qtp1268612900-59657) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[33]} 0 0
   [junit4]   2> 2026961 INFO  (qtp1268612900-59657) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[34]} 0 0
   [junit4]   2> 2026962 INFO  (qtp1268612900-59657) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[35]} 0 0
   [junit4]   2> 2026964 INFO  (qtp1268612900-59654) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[36]} 0 0
   [junit4]   2> 2026966 INFO  (qtp1268612900-59654) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[37]} 0 0
   [junit4]   2> 2026967 INFO  (qtp1268612900-59654) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[38]} 0 0
   [junit4]   2> 2026969 INFO  (qtp1268612900-59656) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[39]} 0 0
   [junit4

[...truncated too long message...]

exception-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-journal-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-normalization-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-operational-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-referral-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-schema-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-subtree-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-trigger-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-jdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-jdbm1-2.0.0-M2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-kerberos-codec-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-ldif-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-mavibot-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-protocol-kerberos-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-protocol-ldap-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-protocol-shared-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-xdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/api-all-1.0.0-M20.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/bcprov-jdk15-1.45.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/cglib-nodep-2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/commons-collections-3.2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/commons-math3-3.4.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/easymock-3.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/ehcache-core-2.4.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/hadoop-common-2.7.2-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/hadoop-hdfs-2.7.2-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/hadoop-minikdc-2.7.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/htrace-core-3.2.0-incubating.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jackson-annotations-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jackson-core-asl-1.9.13.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jackson-databind-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jackson-mapper-asl-1.9.13.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jersey-core-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jersey-server-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jetty-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jetty-util-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/mina-core-2.0.0-M5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/mockito-core-1.9.5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/netty-all-4.0.36.Final.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/objenesis-1.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/presto-parser-0.122.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/slice-0.10.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/analysis/icu/lucene-analyzers-icu-6.2.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/contrib/solr-analysis-extras/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/queryparser/classes/test:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/contrib/analysis-extras/lib/icu4j-56.1.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-launcher.jar:/x1/jenkins/.ant/lib/ivy-2.3.0.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-antlr.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-regexp.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-resolver.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-net.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jmf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-oro.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-xalan2.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-logging.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jsch.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit4.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-javamail.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jai.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jdepend.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bsf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-swing.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-netrexx.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bcel.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-testutil.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-log4j.jar:/x1/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.8/lib/tools.jar:/x1/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.3.4.jar com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/temp/junit4-J2-20160802_174303_058.events @/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/temp/junit4-J2-20160802_174303_058.suites -stdin
   [junit4] ERROR: JVM J2 ended with an exception: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1501)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:121)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:955)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:952)
   [junit4] 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4] 	at java.lang.Thread.run(Thread.java:745)

BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:771: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:707: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/common-build.xml:536: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/common-build.xml:1443: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/common-build.xml:984: At least one slave process threw an exception, first: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.

Total time: 615 minutes 40 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any




[JENKINS] Lucene-Solr-NightlyTests-6.x - Build # 133 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-6.x/133/

4 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest

Error Message:
ObjectTracker found 1 object(s) that were not released!!! [HdfsTransactionLog]

Stack Trace:
java.lang.AssertionError: ObjectTracker found 1 object(s) that were not released!!! [HdfsTransactionLog]
	at __randomizedtesting.SeedInfo.seed([798AF5306665F37A]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.junit.Assert.assertNull(Assert.java:551)
	at org.apache.solr.SolrTestCaseJ4.teardownTestCases(SolrTestCaseJ4.java:258)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:834)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  junit.framework.TestSuite.org.apache.solr.update.TestAtomicUpdateErrorCases

Error Message:


Stack Trace:
java.lang.AssertionError
	at org.apache.lucene.util.TestRuleTemporaryFilesCleanup.before(TestRuleTemporaryFilesCleanup.java:112)
	at com.carrotsearch.randomizedtesting.rules.TestRuleAdapter$1.before(TestRuleAdapter.java:26)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:35)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  junit.framework.TestSuite.org.apache.solr.update.TestAtomicUpdateErrorCases

Error Message:
176 threads leaked from SUITE scope at org.apache.solr.update.TestAtomicUpdateErrorCases:     1) Thread[id=56032, name=coreZkRegister-2064-thread-11, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)    2) Thread[id=56050, name=coreZkRegister-2257-thread-13, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)    3) Thread[id=56110, name=watches-741-thread-24, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)    4) Thread[id=56046, name=coreZkRegister-2257-thread-9, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)    5) Thread[id=56219, name=qtp1537330087-56219, state=WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)         at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870)         at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199)         at java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(ReentrantLock.java:209)         at java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:285)         at org.apache.solr.update.DirectUpdateHandler2.commit(DirectUpdateHandler2.java:575)         at org.apache.solr.update.processor.RunUpdateProcessor.processCommit(RunUpdateProcessorFactory.java:93)         at org.apache.solr.update.processor.UpdateRequestProcessor.processCommit(UpdateRequestProcessor.java:61)         at org.apache.solr.update.processor.DistributedUpdateProcessor.doLocalCommit(DistributedUpdateProcessor.java:1646)         at org.apache.solr.update.processor.DistributedUpdateProcessor.processCommit(DistributedUpdateProcessor.java:1623)         at org.apache.solr.update.processor.LogUpdateProcessorFactory$LogUpdateProcessor.processCommit(LogUpdateProcessorFactory.java:160)         at org.apache.solr.handler.RequestHandlerUtils.handleCommit(RequestHandlerUtils.java:68)         at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:62)         at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:154)         at org.apache.solr.core.SolrCore.execute(SolrCore.java:2089)         at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:652)         at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:459)         at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:257)         at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:208)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676)         at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:108)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676)         at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:581)         at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)         at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1160)         at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511)         at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)         at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1092)         at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)         at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:399)         at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)         at org.eclipse.jetty.server.Server.handle(Server.java:518)         at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:308)         at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:244)         at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)         at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)         at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:186)         at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)         at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)         at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:246)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:156)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)         at java.lang.Thread.run(Thread.java:745)    6) Thread[id=55982, name=zkCallback-740-thread-27, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)    7) Thread[id=56081, name=watches-722-thread-14, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)    8) Thread[id=56103, name=watches-741-thread-23, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)    9) Thread[id=56016, name=coreZkRegister-2053-thread-3, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   10) Thread[id=56229, name=qtp1537330087-56229, state=WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)         at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870)         at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199)         at java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(ReentrantLock.java:209)         at java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:285)         at org.apache.solr.update.DirectUpdateHandler2.commit(DirectUpdateHandler2.java:575)         at org.apache.solr.update.processor.RunUpdateProcessor.processCommit(RunUpdateProcessorFactory.java:93)         at org.apache.solr.update.processor.UpdateRequestProcessor.processCommit(UpdateRequestProcessor.java:61)         at org.apache.solr.update.processor.DistributedUpdateProcessor.doLocalCommit(DistributedUpdateProcessor.java:1646)         at org.apache.solr.update.processor.DistributedUpdateProcessor.processCommit(DistributedUpdateProcessor.java:1623)         at org.apache.solr.update.processor.LogUpdateProcessorFactory$LogUpdateProcessor.processCommit(LogUpdateProcessorFactory.java:160)         at org.apache.solr.handler.RequestHandlerUtils.handleCommit(RequestHandlerUtils.java:68)         at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:62)         at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:154)         at org.apache.solr.core.SolrCore.execute(SolrCore.java:2089)         at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:652)         at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:459)         at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:257)         at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:208)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676)         at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:108)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676)         at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:581)         at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)         at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1160)         at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511)         at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)         at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1092)         at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)         at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:399)         at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)         at org.eclipse.jetty.server.Server.handle(Server.java:518)         at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:308)         at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:244)         at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)         at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)         at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:186)         at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)         at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)         at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:246)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:156)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)         at java.lang.Thread.run(Thread.java:745)   11) Thread[id=56014, name=OverseerCollectionConfigSetProcessor-96341220247732282-127.0.0.1:54687_-n_0000000006, state=TIMED_WAITING, group=Overseer collection creation process.]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.apache.solr.cloud.DistributedQueue.peekElements(DistributedQueue.java:343)         at org.apache.solr.cloud.OverseerTaskQueue.peekTopN(OverseerTaskQueue.java:235)         at org.apache.solr.cloud.OverseerTaskProcessor.run(OverseerTaskProcessor.java:224)         at java.lang.Thread.run(Thread.java:745)   12) Thread[id=56040, name=coreZkRegister-2031-thread-7, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   13) Thread[id=55997, name=zkCallback-740-thread-33-processing-n:127.0.0.1:59829_-EventThread, state=WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:494)   14) Thread[id=55993, name=zkCallback-721-thread-19-processing-n:127.0.0.1:41992_-SendThread(127.0.0.1:45397), state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:349)         at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)   15) Thread[id=56101, name=watches-781-thread-16, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   16) Thread[id=56217, name=qtp1537330087-56217, state=WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at java.lang.Object.wait(Native Method)         at java.lang.Object.wait(Object.java:502)         at org.apache.hadoop.ipc.Client.call(Client.java:1466)         at org.apache.hadoop.ipc.Client.call(Client.java:1412)         at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:229)         at com.sun.proxy.$Proxy51.getFileInfo(Unknown Source)         at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getFileInfo(ClientNamenodeProtocolTranslatorPB.java:771)         at sun.reflect.GeneratedMethodAccessor26.invoke(Unknown Source)         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)         at java.lang.reflect.Method.invoke(Method.java:498)         at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:191)         at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)         at com.sun.proxy.$Proxy55.getFileInfo(Unknown Source)         at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:2108)         at org.apache.hadoop.hdfs.DistributedFileSystem$22.doCall(DistributedFileSystem.java:1305)         at org.apache.hadoop.hdfs.DistributedFileSystem$22.doCall(DistributedFileSystem.java:1301)         at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)         at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1317)         at org.apache.solr.store.hdfs.HdfsDirectory$HdfsIndexInput.<init>(HdfsDirectory.java:212)         at org.apache.solr.store.hdfs.HdfsDirectory.openInput(HdfsDirectory.java:139)         at org.apache.lucene.store.NRTCachingDirectory.openInput(NRTCachingDirectory.java:192)         at org.apache.solr.core.SolrCore.getNewIndexDir(SolrCore.java:327)         at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1577)         at org.apache.solr.update.UpdateLog.openRealtimeSearcher(UpdateLog.java:517)         at org.apache.solr.update.IndexFingerprint.getFingerprint(IndexFingerprint.java:83)         at org.apache.solr.handler.component.RealTimeGetComponent.processGetVersions(RealTimeGetComponent.java:626)         at org.apache.solr.handler.component.RealTimeGetComponent.process(RealTimeGetComponent.java:117)         at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:295)         at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:154)         at org.apache.solr.core.SolrCore.execute(SolrCore.java:2089)         at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:652)         at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:459)         at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:257)         at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:208)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676)         at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:108)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676)         at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:581)         at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)         at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1160)         at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511)         at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)         at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1092)         at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)         at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:399)         at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)         at org.eclipse.jetty.server.Server.handle(Server.java:518)         at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:308)         at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:244)         at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)         at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)         at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:186)         at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)         at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)         at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:246)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:156)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)         at java.lang.Thread.run(Thread.java:745)   17) Thread[id=55972, name=zkCallback-747-thread-8, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   18) Thread[id=55989, name=zkCallback-740-thread-34, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   19) Thread[id=56117, name=recoveryExecutor-778-thread-13-processing-n:127.0.0.1:54687_ x:collection1 s:shard1 c:collection1 r:core_node2, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at java.net.SocketInputStream.socketRead0(Native Method)         at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)         at java.net.SocketInputStream.read(SocketInputStream.java:170)         at java.net.SocketInputStream.read(SocketInputStream.java:141)         at sun.security.ssl.InputRecord.readFully(InputRecord.java:465)         at sun.security.ssl.InputRecord.read(InputRecord.java:503)         at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:973)         at sun.security.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:930)         at sun.security.ssl.AppInputStream.read(AppInputStream.java:105)         at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:160)         at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:84)         at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:273)         at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)         at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)         at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)         at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)         at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)         at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197)         at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)         at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)         at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685)         at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487)         at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)         at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)         at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)         at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)         at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:495)         at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:259)         at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:248)         at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:149)         at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:166)         at org.apache.solr.cloud.RecoveryStrategy.commitOnLeader(RecoveryStrategy.java:201)         at org.apache.solr.cloud.RecoveryStrategy.replicate(RecoveryStrategy.java:141)         at org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:408)         at org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:221)         at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)         at java.util.concurrent.FutureTask.run(FutureTask.java:266)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$31/1587311309.run(Unknown Source)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   20) Thread[id=55978, name=zkCallback-740-thread-23, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   21) Thread[id=56049, name=coreZkRegister-2257-thread-12, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   22) Thread[id=56090, name=watches-741-thread-22, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   23) Thread[id=56026, name=coreZkRegister-2064-thread-5, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   24) Thread[id=55990, name=zkCallback-780-thread-30, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   25) Thread[id=56067, name=watches-741-thread-20, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   26) Thread[id=56070, name=watches-734-thread-13, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   27) Thread[id=55986, name=zkCallback-740-thread-31, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   28) Thread[id=56073, name=coreZkRegister-2053-thread-7, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   29) Thread[id=56004, name=watches-734-thread-11, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   30) Thread[id=56105, name=watches-722-thread-18, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   31) Thread[id=56021, name=OverseerThreadFactory-2837-thread-5, state=WAITING, group=Overseer collection creation process.]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   32) Thread[id=55985, name=zkCallback-740-thread-30, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   33) Thread[id=55965, name=zkCallback-780-thread-24, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   34) Thread[id=56005, name=watches-741-thread-13, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   35) Thread[id=55974, name=zkCallback-747-thread-10, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   36) Thread[id=56212, name=qtp2052259692-56212, state=WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)         at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870)         at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199)         at java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(ReentrantLock.java:209)         at java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:285)         at org.apache.solr.update.DirectUpdateHandler2.commit(DirectUpdateHandler2.java:575)         at org.apache.solr.update.processor.RunUpdateProcessor.processCommit(RunUpdateProcessorFactory.java:93)         at org.apache.solr.update.processor.UpdateRequestProcessor.processCommit(UpdateRequestProcessor.java:61)         at org.apache.solr.update.processor.DistributedUpdateProcessor.doLocalCommit(DistributedUpdateProcessor.java:1646)         at org.apache.solr.update.processor.DistributedUpdateProcessor.processCommit(DistributedUpdateProcessor.java:1623)         at org.apache.solr.update.processor.LogUpdateProcessorFactory$LogUpdateProcessor.processCommit(LogUpdateProcessorFactory.java:160)         at org.apache.solr.handler.RequestHandlerUtils.handleCommit(RequestHandlerUtils.java:68)         at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:62)         at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:154)         at org.apache.solr.core.SolrCore.execute(SolrCore.java:2089)         at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:652)         at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:459)         at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:257)         at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:208)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676)         at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:108)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676)         at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:581)         at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)         at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1160)         at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511)         at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)         at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1092)         at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)         at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:399)         at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)         at org.eclipse.jetty.server.Server.handle(Server.java:518)         at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:308)         at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:244)         at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)         at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)         at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:186)         at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)         at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)         at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:246)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:156)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)         at java.lang.Thread.run(Thread.java:745)   37) Thread[id=56211, name=qtp2052259692-56211, state=WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at java.lang.Object.wait(Native Method)         at java.lang.Object.wait(Object.java:502)         at org.apache.hadoop.ipc.Client.call(Client.java:1466)         at org.apache.hadoop.ipc.Client.call(Client.java:1412)         at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:229)         at com.sun.proxy.$Proxy51.getFileInfo(Unknown Source)         at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getFileInfo(ClientNamenodeProtocolTranslatorPB.java:771)         at sun.reflect.GeneratedMethodAccessor26.invoke(Unknown Source)         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)         at java.lang.reflect.Method.invoke(Method.java:498)         at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:191)         at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)         at com.sun.proxy.$Proxy55.getFileInfo(Unknown Source)         at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:2108)         at org.apache.hadoop.hdfs.DistributedFileSystem$22.doCall(DistributedFileSystem.java:1305)         at org.apache.hadoop.hdfs.DistributedFileSystem$22.doCall(DistributedFileSystem.java:1301)         at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)         at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1317)         at org.apache.solr.store.hdfs.HdfsDirectory$HdfsIndexInput.<init>(HdfsDirectory.java:212)         at org.apache.solr.store.hdfs.HdfsDirectory.openInput(HdfsDirectory.java:139)         at org.apache.lucene.store.NRTCachingDirectory.openInput(NRTCachingDirectory.java:192)         at org.apache.solr.core.SolrCore.getNewIndexDir(SolrCore.java:327)         at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1577)         at org.apache.solr.update.UpdateLog.openRealtimeSearcher(UpdateLog.java:517)         at org.apache.solr.update.IndexFingerprint.getFingerprint(IndexFingerprint.java:83)         at org.apache.solr.handler.component.RealTimeGetComponent.processGetVersions(RealTimeGetComponent.java:626)         at org.apache.solr.handler.component.RealTimeGetComponent.process(RealTimeGetComponent.java:117)         at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:295)         at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:154)         at org.apache.solr.core.SolrCore.execute(SolrCore.java:2089)         at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:652)         at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:459)         at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:257)         at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:208)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676)         at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:108)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676)         at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:581)         at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)         at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1160)         at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511)         at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)         at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1092)         at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)         at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:399)         at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)         at org.eclipse.jetty.server.Server.handle(Server.java:518)         at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:308)         at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:244)         at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)         at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)         at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:186)         at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)         at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)         at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:246)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:156)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)         at java.lang.Thread.run(Thread.java:745)   38) Thread[id=56114, name=updateExecutor-777-thread-13-processing-n:127.0.0.1:54687_ x:collection1 s:shard1 c:collection1 r:core_node2, state=WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.FutureTask.awaitDone(FutureTask.java:429)         at java.util.concurrent.FutureTask.get(FutureTask.java:191)         at org.apache.solr.update.DefaultSolrCoreState$2.run(DefaultSolrCoreState.java:317)         at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)         at java.util.concurrent.FutureTask.run(FutureTask.java:266)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$31/1587311309.run(Unknown Source)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   39) Thread[id=56034, name=zkCallback-733-thread-33, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   40) Thread[id=56054, name=coreZkRegister-2053-thread-4, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   41) Thread[id=56215, name=qtp1537330087-56215, state=WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)         at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870)         at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199)         at java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(ReentrantLock.java:209)         at java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:285)         at org.apache.solr.update.DirectUpdateHandler2.commit(DirectUpdateHandler2.java:575)         at org.apache.solr.update.processor.RunUpdateProcessor.processCommit(RunUpdateProcessorFactory.java:93)         at org.apache.solr.update.processor.UpdateRequestProcessor.processCommit(UpdateRequestProcessor.java:61)         at org.apache.solr.update.processor.DistributedUpdateProcessor.doLocalCommit(DistributedUpdateProcessor.java:1646)         at org.apache.solr.update.processor.DistributedUpdateProcessor.processCommit(DistributedUpdateProcessor.java:1623)         at org.apache.solr.update.processor.LogUpdateProcessorFactory$LogUpdateProcessor.processCommit(LogUpdateProcessorFactory.java:160)         at org.apache.solr.handler.RequestHandlerUtils.handleCommit(RequestHandlerUtils.java:68)         at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:62)         at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:154)         at org.apache.solr.core.SolrCore.execute(SolrCore.java:2089)         at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:652)         at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:459)         at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:257)         at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:208)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676)         at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:108)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676)         at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:581)         at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)         at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1160)         at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511)         at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)         at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1092)         at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)         at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:399)         at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)         at org.eclipse.jetty.server.Server.handle(Server.java:518)         at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:308)         at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:244)         at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)         at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)         at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:186)         at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)         at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)         at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:246)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:156)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)         at java.lang.Thread.run(Thread.java:745)   42) Thread[id=56015, name=OverseerHdfsCoreFailoverThread-96341220247732282-127.0.0.1:54687_-n_0000000006, state=TIMED_WAITING, group=Overseer Hdfs SolrCore Failover Thread.]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.cloud.OverseerAutoReplicaFailoverThread.run(OverseerAutoReplicaFailoverThread.java:137)         at java.lang.Thread.run(Thread.java:745)   43) Thread[id=55995, name=IPC Client (1543981367) connection to localhost/127.0.0.1:44470 from jenkins, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335)         at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157)         at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161)         at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131)         at java.io.FilterInputStream.read(FilterInputStream.java:133)         at java.io.FilterInputStream.read(FilterInputStream.java:133)         at org.apache.hadoop.ipc.Client$Connection$PingInputStream.read(Client.java:520)         at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)         at java.io.BufferedInputStream.read(BufferedInputStream.java:265)         at java.io.DataInputStream.readInt(DataInputStream.java:387)         at org.apache.hadoop.ipc.Client$Connection.receiveRpcResponse(Client.java:1084)         at org.apache.hadoop.ipc.Client$Connection.run(Client.java:979)   44) Thread[id=56042, name=coreZkRegister-2031-thread-9, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   45) Thread[id=56142, name=recoveryExecutor-719-thread-8-processing-n:127.0.0.1:41992_ x:awholynewcollection_1_shard2_replica1 s:shard2 c:awholynewcollection_1 r:core_node9, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at java.net.SocketInputStream.socketRead0(Native Method)         at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)         at java.net.SocketInputStream.read(SocketInputStream.java:170)         at java.net.SocketInputStream.read(SocketInputStream.java:141)         at sun.security.ssl.InputRecord.readFully(InputRecord.java:465)         at sun.security.ssl.InputRecord.read(InputRecord.java:503)         at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:973)         at sun.security.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:930)         at sun.security.ssl.AppInputStream.read(AppInputStream.java:105)         at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:160)         at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:84)         at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:273)         at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)         at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)         at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)         at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)         at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)         at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197)         at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)         at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)         at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685)         at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487)         at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)         at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)         at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)         at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)         at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:495)         at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:259)         at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:248)         at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:149)         at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:166)         at org.apache.solr.cloud.RecoveryStrategy.commitOnLeader(RecoveryStrategy.java:201)         at org.apache.solr.cloud.RecoveryStrategy.replicate(RecoveryStrategy.java:141)         at org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:408)         at org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:221)         at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)         at java.util.concurrent.FutureTask.run(FutureTask.java:266)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$31/1587311309.run(Unknown Source)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   46) Thread[id=56033, name=coreZkRegister-2064-thread-12, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   47) Thread[id=56148, name=watches-734-thread-20, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   48) Thread[id=56189, name=recoveryExecutor-738-thread-5-processing-n:127.0.0.1:59829_ x:awholynewcollection_1_shard3_replica3 s:shard3 c:awholynewcollection_1 r:core_node2, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at java.net.SocketInputStream.socketRead0(Native Method)         at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)         at java.net.SocketInputStream.read(SocketInputStream.java:170)         at java.net.SocketInputStream.read(SocketInputStream.java:141)         at sun.security.ssl.InputRecord.readFully(InputRecord.java:465)         at sun.security.ssl.InputRecord.read(InputRecord.java:503)         at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:973)         at sun.security.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:930)         at sun.security.ssl.AppInputStream.read(AppInputStream.java:105)         at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:160)         at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:84)         at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:273)         at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)         at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)         at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)         at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)         at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)         at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197)         at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)         at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)         at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685)         at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487)         at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)         at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)         at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)         at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)         at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:495)         at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:259)         at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:248)         at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:149)         at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:166)         at org.apache.solr.cloud.RecoveryStrategy.commitOnLeader(RecoveryStrategy.java:201)         at org.apache.solr.cloud.RecoveryStrategy.replicate(RecoveryStrategy.java:141)         at org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:408)         at org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:221)         at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)         at java.util.concurrent.FutureTask.run(FutureTask.java:266)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$31/1587311309.run(Unknown Source)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   49) Thread[id=56149, name=watches-722-thread-21, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   50) Thread[id=56235, name=qtp2052259692-56235, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:546)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:609)         at java.lang.Thread.run(Thread.java:745)   51) Thread[id=56023, name=coreZkRegister-2064-thread-2, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   52) Thread[id=55964, name=zkCallback-780-thread-23, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   53) Thread[id=56083, name=watches-722-thread-15, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   54) Thread[id=56025, name=coreZkRegister-2064-thread-4, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   55) Thread[id=56146, name=recoveryExecutor-731-thread-8-processing-n:127.0.0.1:41170_ x:awholynewstresscollection_collection1_0_shard4_replica1 s:shard4 c:awholynewstresscollection_collection1_0 r:core_node7, state=BLOCKED, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at org.apache.solr.update.UpdateLog.getRecentUpdates(UpdateLog.java:1082)         at org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:252)         at org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:221)         at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)         at java.util.concurrent.FutureTask.run(FutureTask.java:266)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$31/1587311309.run(Unknown Source)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   56) Thread[id=56011, name=zkCallback-733-thread-32, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   57) Thread[id=56193, name=updateExecutor-730-thread-11-processing-n:127.0.0.1:41170_ x:awholynewcollection_2_shard1_replica4 s:shard1 c:awholynewcollection_2 r:core_node3, state=WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.FutureTask.awaitDone(FutureTask.java:429)         at java.util.concurrent.FutureTask.get(FutureTask.java:191)         at org.apache.solr.update.DefaultSolrCoreState$2.run(DefaultSolrCoreState.java:317)         at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)         at java.util.concurrent.FutureTask.run(FutureTask.java:266)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$31/1587311309.run(Unknown Source)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   58) Thread[id=56095, name=coreZkRegister-2053-thread-11, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   59) Thread[id=56024, name=coreZkRegister-2064-thread-3, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   60) Thread[id=56001, name=qtp1586931699-56001, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:233)         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:181)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:171)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:156)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)         at java.lang.Thread.run(Thread.java:745)   61) Thread[id=56039, name=coreZkRegister-2031-thread-6, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)   

[...truncated too long message...]

e-lucene-core:


-check-git-state:


-git-cleanroot:


-copy-git-state:


git-autoclean:


ivy-availability-check:


ivy-fail:


ivy-configure:

[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/top-level-ivy-settings.xml


resolve:


init:


-clover.disable:


-clover.load:


-clover.classpath:


-clover.setup:


clover:


compile-core:


compile-codecs:


-check-git-state:


-git-cleanroot:


-copy-git-state:


git-autoclean:


ivy-availability-check:


ivy-fail:


ivy-configure:

[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/top-level-ivy-settings.xml


resolve:


common.init:


compile-lucene-core:


init:


-clover.disable:


-clover.load:


-clover.classpath:


-clover.setup:


clover:


compile-core:


-clover.disable:


-clover.load:


-clover.classpath:


-clover.setup:


clover:


common.compile-core:


compile-core:


compile-core:


compile-solr-core:


compile-solrj:


check-lucene-core-uptodate:


jar-lucene-core:


check-backward-codecs-uptodate:


jar-backward-codecs:


check-analyzers-phonetic-uptodate:


jar-analyzers-phonetic:


check-analyzers-kuromoji-uptodate:


jar-analyzers-kuromoji:


check-codecs-uptodate:


jar-codecs:


check-expressions-uptodate:


jar-expressions:


check-suggest-uptodate:


jar-suggest:


check-highlighter-uptodate:


jar-highlighter:


check-memory-uptodate:


jar-memory:


check-misc-uptodate:


jar-misc:


check-spatial-extras-uptodate:


jar-spatial-extras:


check-grouping-uptodate:


jar-grouping:


check-queries-uptodate:


jar-queries:


check-queryparser-uptodate:


jar-queryparser:


check-join-uptodate:


jar-join:


check-sandbox-uptodate:


jar-sandbox:


check-classification-uptodate:


jar-classification:


prep-lucene-jars:


resolve-example:


ivy-availability-check:


ivy-fail:


ivy-configure:

[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/top-level-ivy-settings.xml


resolve:


resolve-server:


ivy-availability-check:


ivy-fail:


ivy-fail:


ivy-configure:

[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/top-level-ivy-settings.xml


resolve:


-check-git-state:


-git-cleanroot:


-copy-git-state:


git-autoclean:


ivy-availability-check:


ivy-fail:


ivy-fail:


ivy-configure:

[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/top-level-ivy-settings.xml


resolve:


common.init:


compile-lucene-core:


-check-git-state:


-git-cleanroot:


-copy-git-state:


git-autoclean:


ivy-availability-check:


ivy-fail:


ivy-configure:

[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/top-level-ivy-settings.xml


resolve:


init:


-clover.disable:


-clover.load:


-clover.classpath:


-clover.setup:


clover:


compile-core:


init:


-clover.disable:


-clover.load:


-clover.classpath:


-clover.setup:


clover:


common.compile-core:


compile-core:


check-lucene-core-uptodate:


jar-lucene-core:


check-backward-codecs-uptodate:


jar-backward-codecs:


check-analyzers-phonetic-uptodate:


jar-analyzers-phonetic:


check-analyzers-kuromoji-uptodate:


jar-analyzers-kuromoji:


check-codecs-uptodate:


jar-codecs:


check-expressions-uptodate:


jar-expressions:


check-suggest-uptodate:


jar-suggest:


check-highlighter-uptodate:


jar-highlighter:


check-memory-uptodate:


jar-memory:


check-misc-uptodate:


jar-misc:


check-spatial-extras-uptodate:


jar-spatial-extras:


check-grouping-uptodate:


jar-grouping:


check-queries-uptodate:


jar-queries:


check-queryparser-uptodate:


jar-queryparser:


check-join-uptodate:


jar-join:


check-sandbox-uptodate:


jar-sandbox:


check-classification-uptodate:


jar-classification:


prep-lucene-jars:


resolve-example:


ivy-availability-check:


ivy-fail:


ivy-configure:

[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/top-level-ivy-settings.xml


resolve:


resolve-server:


ivy-availability-check:


ivy-fail:


ivy-fail:


ivy-configure:

[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/top-level-ivy-settings.xml


resolve:


-check-git-state:


-git-cleanroot:


-copy-git-state:


git-autoclean:


ivy-fail:


resolve:


common.init:


compile-lucene-core:


-check-git-state:


-git-cleanroot:


-copy-git-state:


git-autoclean:


ivy-availability-check:


ivy-fail:


ivy-configure:

[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/top-level-ivy-settings.xml


resolve:


init:


-clover.disable:


-clover.load:


-clover.classpath:


-clover.setup:


clover:


compile-core:


init:


common.compile-core:

    [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/contrib/solr-velocity/classes/java

    [javac] Compiling 5 source files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/contrib/solr-velocity/classes/java

    [javac] Note: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/contrib/velocity/src/java/org/apache/solr/response/VelocityResponseWriter.java uses or overrides a deprecated API.

    [javac] Note: Recompile with -Xlint:deprecation for details.

    [javac] Note: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/contrib/velocity/src/java/org/apache/solr/response/VelocityResponseWriter.java uses unchecked or unsafe operations.

    [javac] Note: Recompile with -Xlint:unchecked for details.

     [copy] Copying 13 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/contrib/solr-velocity/classes/java


common-solr.compile-core:


compile-core:


compile-test-framework:


-check-git-state:


-git-cleanroot:


-copy-git-state:


git-autoclean:


ivy-availability-check:


ivy-fail:


ivy-configure:

[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/top-level-ivy-settings.xml


resolve:


init:


compile-lucene-core:


compile-codecs:


-check-git-state:


-git-cleanroot:


-copy-git-state:


git-autoclean:


ivy-availability-check:


ivy-fail:


ivy-configure:

[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/top-level-ivy-settings.xml


resolve:


common.init:


compile-lucene-core:


init:


-clover.disable:


-clover.load:


-clover.classpath:


-clover.setup:


clover:


compile-core:


-clover.disable:


-clover.load:


-clover.classpath:


-clover.setup:


clover:


common.compile-core:


compile-core:


common.compile-test:

    [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/contrib/solr-velocity/classes/test

    [javac] Compiling 2 source files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/contrib/solr-velocity/classes/test

     [copy] Copying 2 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/contrib/solr-velocity/classes/test


compile-test:


install-junit4-taskdef:


compile-tools:


-check-git-state:


-git-cleanroot:


-copy-git-state:


git-autoclean:


ivy-availability-check:


ivy-fail:


ivy-configure:

[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/top-level-ivy-settings.xml


resolve:


init:


compile-core:


validate:


resolve-groovy:


-init-totals:


-test:

    [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/contrib/solr-velocity/test

[loadresource] Do not set property tests.explicitclass as its length is 0.

    [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/contrib/solr-velocity/test/temp

    [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/.caches/test-stats/solr-velocity

   [junit4] <JUnit4> says ¡Hola! Master seed: CFC9244F129B0874

   [junit4] Executing 1 suite with 1 JVM.

   [junit4] 

   [junit4] Started J0 PID(12950@lucene1-us-west).

   [junit4] Suite: org.apache.solr.velocity.VelocityResponseWriterTest

   [junit4] Completed [1/1] in 2.27s, 13 tests

   [junit4] 

   [junit4] JVM J0:     0.29 ..     3.15 =     2.85s

   [junit4] Execution time total: 3.15 sec.

   [junit4] Tests summary: 1 suite, 13 tests

   [junit4] Could not remove temporary path: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/contrib/solr-velocity/test/J0 (/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/contrib/solr-velocity/test/J0)

     [echo] 5 slowest tests:

[junit4:tophints]   2.27s | org.apache.solr.velocity.VelocityResponseWriterTest


-check-totals:


common.test:


-check-totals:


test:


ivy-availability-check:


ivy-configure:

[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/top-level-ivy-settings.xml


resolve-groovy:


-check-totals:


BUILD FAILED

/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:771: The following error occurred while executing this line:

/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:715: Some of the tests produced a heap dump, but did not fail. Maybe a suppressed OutOfMemoryError? Dumps created:

* java_pid13866.hprof


Total time: 395 minutes 7 seconds

Build step 'Invoke Ant' marked build as failure

Archiving artifacts

Recording test results

Email was triggered for: Failure - Any

Sending email for trigger: Failure - Any





[JENKINS] Lucene-Solr-NightlyTests-6.x - Build # 132 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-6.x/132/

2 tests failed.
FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Timeout occured while waiting response from server at: https://127.0.0.1:36105

Stack Trace:
org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: https://127.0.0.1:36105
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:601)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:259)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:248)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.makeRequest(CollectionsAPIDistributedZkTest.java:399)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testErrorHandling(CollectionsAPIDistributedZkTest.java:515)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test(CollectionsAPIDistributedZkTest.java:179)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.net.SocketTimeoutException: Read timed out
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
	at java.net.SocketInputStream.read(SocketInputStream.java:170)
	at java.net.SocketInputStream.read(SocketInputStream.java:141)
	at sun.security.ssl.InputRecord.readFully(InputRecord.java:465)
	at sun.security.ssl.InputRecord.read(InputRecord.java:503)
	at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:973)
	at sun.security.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:930)
	at sun.security.ssl.AppInputStream.read(AppInputStream.java:105)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:160)
	at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:84)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:273)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
	at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
	at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)
	at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)
	at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197)
	at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)
	at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)
	at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685)
	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487)
	at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:495)
	... 47 more


FAILED:  junit.framework.TestSuite.org.apache.solr.handler.TestReplicationHandler

Error Message:
ObjectTracker found 4 object(s) that were not released!!! [NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory]

Stack Trace:
java.lang.AssertionError: ObjectTracker found 4 object(s) that were not released!!! [NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory]
	at __randomizedtesting.SeedInfo.seed([AF0C4F5D35A09767]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.junit.Assert.assertNull(Assert.java:551)
	at org.apache.solr.SolrTestCaseJ4.teardownTestCases(SolrTestCaseJ4.java:258)
	at sun.reflect.GeneratedMethodAccessor43.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:834)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
Started by timer

[EnvInject] - Loading node environment variables.

Building remotely on lucene in workspace /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x

 > git rev-parse --is-inside-work-tree # timeout=10

Fetching changes from the remote Git repository

 > git config remote.origin.url https://git-wip-us.apache.org/repos/asf/lucene-solr.git # timeout=10

Cleaning workspace

 > git rev-parse --verify HEAD # timeout=10

Resetting working tree

 > git reset --hard # timeout=10

 > git clean -fdx # timeout=10

Fetching upstream changes from https://git-wip-us.apache.org/repos/asf/lucene-solr.git

 > git --version # timeout=10

 > git -c core.askpass=true fetch --tags --progress https://git-wip-us.apache.org/repos/asf/lucene-solr.git +refs/heads/*:refs/remotes/origin/*

 > git rev-parse refs/remotes/origin/branch_6x^{commit} # timeout=10

 > git rev-parse refs/remotes/origin/origin/branch_6x^{commit} # timeout=10

Checking out Revision e3de51be2edd086c29be8fdcfca7f8a5990a640c (refs/remotes/origin/branch_6x)

 > git config core.sparsecheckout # timeout=10

 > git checkout -f e3de51be2edd086c29be8fdcfca7f8a5990a640c

 > git rev-list e3de51be2edd086c29be8fdcfca7f8a5990a640c # timeout=10

No emails were triggered.

[Lucene-Solr-NightlyTests-6.x] $ /home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/bin/ant -Dtests.multiplier=2 -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt jenkins-nightly

Buildfile: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml


jenkins-nightly:


-print-java-info:

[java-info] java version "1.8.0_74"

[java-info] Java(TM) SE Runtime Environment (1.8.0_74-b02, Oracle Corporation)

[java-info] Java HotSpot(TM) 64-Bit Server VM (25.74-b02, Oracle Corporation)

[java-info] Test args: []


clean:


clean:


clean:


ivy-availability-check:


ivy-fail:


ivy-configure:

[ivy:configure] :: Apache Ivy 2.3.0 - 20130110142753 :: http://ant.apache.org/ivy/ ::

[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/top-level-ivy-settings.xml


resolve-groovy:

[ivy:cachepath] :: resolving dependencies :: org.codehaus.groovy#groovy-all-caller;working

[ivy:cachepath] 	confs: [default]

[ivy:cachepath] 	found org.codehaus.groovy#groovy-all;2.4.6 in public

[ivy:cachepath] :: resolution report :: resolve 88ms :: artifacts dl 1ms

	---------------------------------------------------------------------

	|                  |            modules            ||   artifacts   |

	|       conf       | number| search|dwnlded|evicted|| number|dwnlded|

	---------------------------------------------------------------------

	|      default     |   1   |   0   |   0   |   0   ||   1   |   0   |

	---------------------------------------------------------------------


-test-with-heapdumps-enabled:

     [echo] Java HotSpot(TM) 64-Bit Server VM: Enabling heap dumps on OutOfMemoryError to dir '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/heapdumps'.

    [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/heapdumps


ivy-availability-check:


ivy-fail:


ivy-configure:

[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/top-level-ivy-settings.xml


resolve-groovy:


test:


ivy-availability-check:


ivy-fail:


ivy-configure:

[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/top-level-ivy-settings.xml


-clover.load:


resolve-groovy:


-init-totals:


test-core:


-clover.disable:


ivy-availability-check:


ivy-fail:


ivy-configure:

[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/top-level-ivy-settings.xml


-clover.load:


-clover.classpath:


-clover.setup:


clover:


-check-git-state:


-git-cleanroot:


-copy-git-state:

     [copy] Copying 1 file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build


git-autoclean:


resolve:


init:


compile-core:

    [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java

    [javac] Compiling 782 source files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java

    [javac] Note: Some input files use or override a deprecated API.

    [javac] Note: Recompile with -Xlint:deprecation for details.

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java/org/apache/lucene/document/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java/org/apache/lucene/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java/org/apache/lucene/util/bkd/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java/org/apache/lucene/codecs/compressing/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java/org/apache/lucene/codecs/lucene60/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java/org/apache/lucene/codecs/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java/org/apache/lucene/util/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java/org/apache/lucene/index/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java/org/apache/lucene/util/fst/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java/org/apache/lucene/analysis/tokenattributes/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java/org/apache/lucene/search/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java/org/apache/lucene/codecs/lucene62/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java/org/apache/lucene/util/packed/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java/org/apache/lucene/util/mutable/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java/org/apache/lucene/codecs/perfield/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java/org/apache/lucene/analysis/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java/org/apache/lucene/analysis/standard/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java/org/apache/lucene/store/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java/org/apache/lucene/codecs/blocktree/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java/org/apache/lucene/search/spans/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java/org/apache/lucene/codecs/lucene54/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java/org/apache/lucene/search/similarities/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java/org/apache/lucene/util/automaton/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java/org/apache/lucene/codecs/lucene53/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java/org/apache/lucene/codecs/lucene50/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java/org/apache/lucene/geo/package-info.class

     [copy] Copying 3 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java


compile-test-framework:


-check-git-state:


-git-cleanroot:


-copy-git-state:


git-autoclean:


ivy-availability-check:


ivy-fail:


ivy-configure:

[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/top-level-ivy-settings.xml


resolve:


init:


compile-lucene-core:


-check-git-state:


-git-cleanroot:


-copy-git-state:


git-autoclean:


ivy-availability-check:


ivy-fail:


ivy-configure:

[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/top-level-ivy-settings.xml


resolve:


init:


-clover.disable:


-clover.load:


-clover.classpath:


-clover.setup:


clover:


compile-core:


compile-codecs:


-check-git-state:


-git-cleanroot:


-copy-git-state:


git-autoclean:


ivy-availability-check:


ivy-fail:


ivy-configure:

[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/top-level-ivy-settings.xml


resolve:


common.init:


compile-lucene-core:


init:


-clover.disable:


-clover.load:


-clover.classpath:


-clover.setup:


clover:


compile-core:

    [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/codecs/classes/java

    [javac] Compiling 66 source files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/codecs/classes/java

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/codecs/classes/java/org/apache/lucene/codecs/bloom/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/codecs/classes/java/org/apache/lucene/codecs/memory/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/codecs/classes/java/org/apache/lucene/codecs/simpletext/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/codecs/classes/java/org/apache/lucene/codecs/blockterms/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/codecs/classes/java/org/apache/lucene/codecs/blocktreeords/package-info.class

     [copy] Copying 3 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/codecs/classes/java


-clover.disable:


-clover.load:


-clover.classpath:


-clover.setup:


clover:


common.compile-core:

    [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/test-framework/classes/java

    [javac] Compiling 185 source files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/test-framework/classes/java

    [javac] Note: Some input files use or override a deprecated API.

    [javac] Note: Recompile with -Xlint:deprecation for details.

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/test-framework/classes/java/org/apache/lucene/codecs/asserting/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/test-framework/classes/java/org/apache/lucene/codecs/ramonly/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/test-framework/classes/java/org/apache/lucene/mockfile/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/test-framework/classes/java/org/apache/lucene/codecs/cheapbastard/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/test-framework/classes/java/org/apache/lucene/codecs/mockrandom/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/test-framework/classes/java/org/apache/lucene/codecs/cranky/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/test-framework/classes/java/org/apache/lucene/codecs/compressing/dummy/package-info.class

     [copy] Copying 4 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/test-framework/classes/java


compile-core:


compile-test:

    [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/test

    [javac] Compiling 445 source files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/test

    [javac] Note: Some input files use or override a deprecated API.

    [javac] Note: Recompile with -Xlint:deprecation for details.

     [copy] Copying 3 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/test


install-junit4-taskdef:


validate:


resolve-groovy:


-init-totals:


-test:

    [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/test

[loadresource] Do not set property tests.explicitclass as its length is 0.

    [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/test/temp

    [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/.caches/test-stats/core

   [junit4] <JUnit4> says שלום! Master seed: 17A3DDEE974B3968

   [junit4] Executing 438 suites with 3 JVMs.

   [junit4] 

   [junit4] Started J0 PID(9183@localhost).

   [junit4] Started J2 PID(9177@localhost).

   [junit4] Started J1 PID(9178@localhost).

   [junit4] Suite: org.apache.lucene.search.TestNot

   [junit4] Completed [1/438] on J2 in 0.89s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestMergedIterator

   [junit4] Completed [2/438] on J2 in 0.61s, 25 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestSortedNumericSortField

   [junit4] Completed [3/438] on J2 in 0.67s, 9 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestSortRandom

   [junit4] Completed [4/438] on J1 in 2.72s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestSetOnce

   [junit4] Completed [5/438] on J1 in 0.19s, 4 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.spans.TestFilterSpans

   [junit4] Completed [6/438] on J1 in 0.06s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestBooleanQueryVisitSubscorers

   [junit4] Completed [7/438] on J1 in 0.51s, 5 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestMathUtil

   [junit4] Completed [8/438] on J1 in 0.10s, 5 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestSumDocFreq

   [junit4] Completed [9/438] on J2 in 2.01s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestSimpleExplanations

   [junit4] Completed [10/438] on J0 in 4.34s, 70 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestNotDocIdSet

   [junit4] Completed [11/438] on J0 in 2.64s, 6 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestDocIdSetBuilder

   [junit4] Completed [12/438] on J2 in 2.77s, 7 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.similarities.TestSimilarity2

   [junit4] Completed [13/438] on J0 in 1.17s, 8 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestCachingCollector

   [junit4] Completed [14/438] on J0 in 0.04s, 4 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestUnicodeUtil

   [junit4] Completed [15/438] on J2 in 1.77s, 5 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.automaton.TestCompiledAutomaton

   [junit4] Completed [16/438] on J2 in 1.05s, 6 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestFixedBitDocIdSet

   [junit4] Completed [17/438] on J0 in 1.85s, 5 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestTimSorter

   [junit4] Completed [18/438] on J0 in 0.27s, 9 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.automaton.TestLevenshteinAutomata

   [junit4] Completed [19/438] on J0 in 4.97s, 4 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestQueryBuilder

   [junit4] Completed [20/438] on J0 in 0.09s, 21 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.Test2BPagedBytes

   [junit4] IGNOR/A 0.01s J0 | Test2BPagedBytes.test

   [junit4]    > Assumption #1: 'monster' test group is disabled (@Monster(value=You must increase heap to > 2 G to run this))

   [junit4] Completed [21/438] on J0 in 0.02s, 1 test, 1 skipped

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestDocCount

   [junit4] Completed [22/438] on J0 in 0.69s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.automaton.TestAutomaton

   [junit4] Completed [23/438] on J1 in 14.40s, 56 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.automaton.TestDeterminizeLexicon

   [junit4] Completed [24/438] on J1 in 2.28s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.packed.TestDirectMonotonic

   [junit4] Completed [25/438] on J0 in 5.51s, 4 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestIntroSorter

   [junit4] Completed [26/438] on J0 in 0.09s, 9 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestTragicIndexWriterDeadlock

   [junit4] Completed [27/438] on J0 in 0.04s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestSloppyMath

   [junit4] Completed [28/438] on J1 in 1.53s, 8 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestCharsRefBuilder

   [junit4] Completed [29/438] on J1 in 0.06s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestDoc

   [junit4] Completed [30/438] on J1 in 0.33s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestStressIndexing

   [junit4] Completed [31/438] on J0 in 4.68s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestCustomNorms

   [junit4] Completed [32/438] on J0 in 7.09s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestTotalHitCountCollector

   [junit4] Completed [33/438] on J0 in 0.06s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestExitableDirectoryReader

   [junit4] Completed [34/438] on J0 in 0.57s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestCloseableThreadLocal

   [junit4] Completed [35/438] on J0 in 0.05s, 3 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestFilterWeight

   [junit4] Completed [36/438] on J0 in 0.05s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestMultiLevelSkipList

   [junit4] Completed [37/438] on J0 in 0.67s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestSmallFloat

   [junit4] Completed [38/438] on J0 in 0.34s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestIndexWriterMergePolicy

   [junit4] Completed [39/438] on J0 in 5.55s, 7 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.codecs.TestCodecUtil

   [junit4] Completed [40/438] on J0 in 0.08s, 15 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestMergeRateLimiter

   [junit4] Completed [41/438] on J0 in 0.04s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestComplexExplanationsOfNonMatches

   [junit4] Completed [42/438] on J0 in 0.22s, 21 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestOfflineSorter

   [junit4] Completed [43/438] on J2 in 31.98s, 14 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.packed.TestPackedInts

   [junit4] Completed [44/438] on J1 in 34.16s, 28 tests, 2 skipped

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestPagedBytes

   [junit4] Completed [45/438] on J1 in 4.16s, 4 tests, 1 skipped

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestTwoPhaseCommitTool

   [junit4] Completed [46/438] on J1 in 0.02s, 3 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.store.TestMultiMMap

   [junit4] Completed [47/438] on J2 in 19.06s, 56 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestTermVectorsWriter

   [junit4] Completed [48/438] on J1 in 0.53s, 15 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestCodecs

   [junit4] Completed [49/438] on J2 in 0.73s, 3 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.mutable.TestMutableValues

   [junit4] Completed [50/438] on J2 in 0.07s, 6 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestIOUtils

   [junit4] Completed [51/438] on J2 in 0.16s, 18 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestReaderWrapperDVTypeCheck

   [junit4] Completed [52/438] on J2 in 0.06s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestAllFilesHaveChecksumFooter

   [junit4] Completed [53/438] on J1 in 1.64s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestOmitTf

   [junit4] Completed [54/438] on J2 in 1.59s, 6 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.spans.TestBasics

   [junit4] Completed [55/438] on J1 in 4.74s, 25 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestBagOfPostings

   [junit4] Completed [56/438] on J2 in 8.16s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestSloppyPhraseQuery

   [junit4] Completed [57/438] on J2 in 3.39s, 9 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestDocInverterPerFieldErrorInfo

   [junit4] Completed [58/438] on J2 in 0.06s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestCollectionUtil

   [junit4] Completed [59/438] on J2 in 16.44s, 4 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestAutomatonQueryUnicode

   [junit4] Completed [60/438] on J2 in 0.07s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestIndexWriterDelete

   [junit4] IGNOR/A 0.00s J0 | TestIndexWriterDelete.testIndexingThenDeleting

   [junit4]    > Assumption #1: This test cannot run with Memory codec

   [junit4] Completed [61/438] on J0 in 59.94s, 27 tests, 1 skipped

   [junit4] 

   [junit4] Suite: org.apache.lucene.store.TestRateLimiter

   [junit4] Completed [62/438] on J0 in 1.47s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestCrash

   [junit4] Completed [63/438] on J0 in 1.22s, 5 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.analysis.TestCachingTokenFilter

   [junit4] Completed [64/438] on J0 in 0.02s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestIndexWriterOnJRECrash

   [junit4] Completed [65/438] on J0 in 4.02s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestLegacyNumericUtils

   [junit4] Completed [66/438] on J0 in 1.39s, 12 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.codecs.lucene50.TestLucene50CompoundFormat

   [junit4] Completed [67/438] on J0 in 4.74s, 26 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestDateSort

   [junit4] Completed [68/438] on J0 in 0.03s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.analysis.tokenattributes.TestCharTermAttributeImpl

   [junit4] Completed [69/438] on J0 in 0.38s, 12 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.automaton.LimitedFiniteStringsIteratorTest

   [junit4] Completed [70/438] on J0 in 0.15s, 6 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.store.TestHugeRamFile

   [junit4] Completed [71/438] on J0 in 1.69s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestReaderClosed

   [junit4] Completed [72/438] on J0 in 0.11s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestBytesRef

   [junit4] Completed [73/438] on J0 in 0.02s, 3 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestPersistentSnapshotDeletionPolicy

   [junit4] Completed [74/438] on J0 in 1.57s, 13 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestPointValues

   [junit4] Completed [75/438] on J0 in 4.42s, 32 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.spans.TestSpansEnum

   [junit4] Completed [76/438] on J0 in 0.12s, 7 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestMatchAllDocsQuery

   [junit4] Completed [77/438] on J0 in 0.06s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestTopFieldCollector

   [junit4] Completed [78/438] on J0 in 0.80s, 7 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestSwappedIndexFiles

   [junit4] Completed [79/438] on J0 in 0.18s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestIndexWriterOnDiskFull

   [junit4] Completed [80/438] on J2 in 35.08s, 4 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestNRTReaderWithThreads

   [junit4] Completed [81/438] on J0 in 3.48s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestSparseFixedBitDocIdSet

   [junit4] Completed [82/438] on J2 in 2.52s, 5 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestFlex

   [junit4] Completed [83/438] on J2 in 0.42s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.bkd.TestDocIdsWriter

   [junit4] Completed [84/438] on J0 in 5.62s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestSizeBoundedForceMerge

   [junit4] Completed [85/438] on J0 in 0.08s, 11 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestManyFields

   [junit4] Completed [86/438] on J0 in 1.37s, 3 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.spans.TestSpanCollection

   [junit4] Completed [87/438] on J0 in 0.02s, 3 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestTermScorer

   [junit4] Completed [88/438] on J0 in 0.11s, 4 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.codecs.compressing.TestFastDecompressionMode

   [junit4] Completed [89/438] on J2 in 5.16s, 11 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.store.TestRAMDirectory

   [junit4] IGNOR/A 0.01s J0 | TestRAMDirectory.testFsyncDoesntCreateNewFiles

   [junit4]    > Assumption #1: test only works for FSDirectory subclasses

   [junit4] IGNOR/A 0.00s J0 | TestRAMDirectory.testPendingDeletions

   [junit4]    > Assumption #1: we can only install VirusCheckingFS on an FSDirectory

   [junit4] Completed [90/438] on J0 in 3.65s, 48 tests, 2 skipped

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestInPlaceMergeSorter

   [junit4] Completed [91/438] on J0 in 0.10s, 9 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestDirectoryReaderReopen

   [junit4] Completed [92/438] on J0 in 3.11s, 14 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestTermQuery

   [junit4] Completed [93/438] on J0 in 0.04s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestIndexWriterMerging

   [junit4] Completed [94/438] on J2 in 8.53s, 6 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestTerms

   [junit4] Completed [95/438] on J2 in 2.55s, 8 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestFilterDirectoryReader

   [junit4] Completed [96/438] on J2 in 0.07s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.store.TestByteArrayDataInput

   [junit4] Completed [97/438] on J2 in 0.03s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestNeverDelete

   [junit4] Completed [98/438] on J0 in 5.72s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.automaton.TestRegExp

   [junit4] Completed [99/438] on J0 in 0.28s, 5 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestAtomicUpdate

   [junit4] Completed [100/438] on J2 in 7.23s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.TestDemo

   [junit4] Completed [101/438] on J2 in 0.05s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestForTooMuchCloning

   [junit4] Completed [102/438] on J2 in 0.36s, 1 test

   [junit4] 

   [junit4] HEARTBEAT J1 PID(9178@localhost): 2016-07-31T05:43:08, stalled for 71.8s at: TestDuelingCodecsAtNight.testBigEquals

   [junit4] Suite: org.apache.lucene.util.TestIntroSelector

   [junit4] Completed [103/438] on J2 in 2.04s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.codecs.lucene50.TestForUtil

   [junit4] Completed [104/438] on J2 in 0.18s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestTieredMergePolicy

   [junit4] Completed [105/438] on J2 in 14.17s, 6 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.analysis.TestWordlistLoader

   [junit4] Completed [106/438] on J2 in 0.04s, 3 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.automaton.FiniteStringsIteratorTest

   [junit4] Completed [107/438] on J2 in 1.18s, 7 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.spans.TestSpanNearQuery

   [junit4] Completed [108/438] on J2 in 0.07s, 4 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestMultiTermsEnum

   [junit4] Completed [109/438] on J2 in 0.03s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.store.TestNativeFSLockFactory

   [junit4] IGNOR/A 0.02s J2 | TestNativeFSLockFactory.testDeleteLockFile

   [junit4]    > Assumption #1: we must be able to delete an open file

   [junit4] IGNOR/A 0.00s J2 | TestNativeFSLockFactory.testStressLocks

   [junit4]    > Assumption #1: cannot handle buggy Files.delete

   [junit4] Completed [110/438] on J2 in 7.20s, 10 tests, 2 skipped

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestMaxTermFrequency

   [junit4] Completed [111/438] on J2 in 0.36s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestMergePolicyWrapper

   [junit4] Completed [112/438] on J2 in 0.04s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.spans.TestSpanBoostQuery

   [junit4] Completed [113/438] on J2 in 0.05s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestBooleanMinShouldMatch

   [junit4] Completed [114/438] on J2 in 4.99s, 17 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestSimpleExplanationsOfNonMatches

   [junit4] Completed [115/438] on J2 in 0.40s, 70 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestParallelReaderEmptyIndex

   [junit4] Completed [116/438] on J2 in 0.04s, 2 tests

   [junit4] 

   [junit4] HEARTBEAT J1 PID(9178@localhost): 2016-07-31T05:44:09, stalled for  132s at: TestDuelingCodecsAtNight.testBigEquals

   [junit4] HEARTBEAT J0 PID(9183@localhost): 2016-07-31T05:44:09, stalled for 68.4s at: TestSearcherManager.testSearcherManager

   [junit4] Suite: org.apache.lucene.index.TestIndexWriterThreadsToSegments

   [junit4] Completed [117/438] on J2 in 61.30s, 4 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.fst.TestBytesStore

   [junit4] Completed [118/438] on J2 in 19.91s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestOmitPositions

   [junit4] Completed [119/438] on J2 in 0.16s, 4 tests

   [junit4] 

   [junit4] HEARTBEAT J1 PID(9178@localhost): 2016-07-31T05:45:09, stalled for  192s at: TestDuelingCodecsAtNight.testBigEquals

   [junit4] HEARTBEAT J0 PID(9183@localhost): 2016-07-31T05:45:09, stalled for  128s at: TestSearcherManager.testSearcherManager

   [junit4] Suite: org.apache.lucene.index.TestIndexWriterMaxDocs

   [junit4] IGNOR/A 0.00s J2 | TestIndexWriterMaxDocs.testExactlyAtTrueLimit

   [junit4]    > Assumption #1: 'monster' test group is disabled (@Monster(value=takes over two hours))

   [junit4] Completed [120/438] on J2 in 14.80s, 21 tests, 1 skipped

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestSynonymQuery

   [junit4] Completed [121/438] on J2 in 0.06s, 4 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestSimilarityProvider

   [junit4] Completed [122/438] on J2 in 0.03s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.store.TestBufferedIndexInput

   [junit4] Completed [123/438] on J2 in 0.43s, 4 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestFrequencyTrackingRingBuffer

   [junit4] Completed [124/438] on J2 in 0.63s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestCodecHoldsOpenFiles

   [junit4] Completed [125/438] on J2 in 0.12s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestReqExclBulkScorer

   [junit4] Completed [126/438] on J2 in 0.11s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestIndexableField

   [junit4] Completed [127/438] on J2 in 1.83s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.bkd.TestBKD

   [junit4] Completed [128/438] on J2 in 30.03s, 16 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestMultiValuedNumericRangeQuery

   [junit4] Completed [129/438] on J2 in 5.08s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestElevationComparator

   [junit4] Completed [130/438] on J2 in 0.10s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.store.TestSimpleFSLockFactory

   [junit4] Completed [131/438] on J2 in 10.80s, 7 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.Test2BBinaryDocValues

   [junit4] IGNOR/A 0.00s J2 | Test2BBinaryDocValues.testFixedBinary

   [junit4]    > Assumption #1: 'monster' test group is disabled (@Monster(value=takes ~ 6 hours if the heap is 5gb))

   [junit4] IGNOR/A 0.00s J2 | Test2BBinaryDocValues.testVariableBinary

   [junit4]    > Assumption #1: 'monster' test group is disabled (@Monster(value=takes ~ 6 hours if the heap is 5gb))

   [junit4] Completed [132/438] on J2 in 0.09s, 2 tests, 2 skipped

   [junit4] 

   [junit4] HEARTBEAT J1 PID(9178@localhost): 2016-07-31T05:46:09, stalled for  252s at: TestDuelingCodecsAtNight.testBigEquals

   [junit4] HEARTBEAT J0 PID(9183@localhost): 2016-07-31T05:46:09, stalled for  188s at: TestSearcherManager.testSearcherManager

   [junit4] Suite: org.apache.lucene.index.TestConsistentFieldNumbers

   [junit4] Completed [133/438] on J2 in 11.07s, 4 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestPriorityQueue

   [junit4] Completed [134/438] on J2 in 0.22s, 9 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestNumericRangeQuery32

   [junit4] Completed [135/438] on J2 in 41.51s, 23 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestLongPostings

   [junit4] Completed [136/438] on J2 in 5.93s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestMultiCollector

   [junit4] Completed [137/438] on J2 in 2.02s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestWildcard

   [junit4] Completed [138/438] on J2 in 0.19s, 8 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestSimilarity

   [junit4] Completed [139/438] on J2 in 0.22s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestPayloads

   [junit4] Completed [140/438] on J2 in 0.33s, 7 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestRecyclingByteBlockAllocator

   [junit4] Completed [141/438] on J2 in 0.08s, 3 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.spans.TestSpanFirstQuery

   [junit4] Completed [142/438] on J2 in 0.17s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestBooleanOr

   [junit4] Completed [143/438] on J2 in 3.41s, 7 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestSort

   [junit4] Completed [144/438] on J2 in 0.59s, 25 tests

   [junit4] 

   [junit4] HEARTBEAT J0 PID(9183@localhost): 2016-07-31T05:47:09, stalled for  249s at: TestSearcherManager.testSearcherManager

   [junit4] HEARTBEAT J1 PID(9178@localhost): 2016-07-31T05:47:09, stalled for  312s at: TestDuelingCodecsAtNight.testBigEquals

   [junit4] HEARTBEAT J0 PID(9183@localhost): 2016-07-31T05:48:09, stalled for  309s at: TestSearcherManager.testSearcherManager

   [junit4] HEARTBEAT J1 PID(9178@localhost): 2016-07-31T05:48:09, stalled for  372s at: TestDuelingCodecsAtNight.testBigEquals

   [junit4] HEARTBEAT J2 PID(9177@localhost): 2016-07-31T05:48:21, stalled for 71.5s at: TestDocumentsWriterStallControl.testAccquireReleaseRace

   [junit4] Suite: org.apache.lucene.search.TestSearcherManager

   [junit4] Completed [145/438] on J0 in 330.36s, 10 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestPhraseQuery

   [junit4] Completed [146/438] on J0 in 1.12s, 21 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestBytesRefArray

   [junit4] Completed [147/438] on J0 in 0.24s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.Test2BPositions

   [junit4] IGNOR/A 0.00s J0 | Test2BPositions.test

   [junit4]    > Assumption #1: 'monster' test group is disabled (@Monster(value=uses lots of space and takes a few minutes))

   [junit4] Completed [148/438] on J0 in 0.00s, 1 test, 1 skipped

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestDocValuesIndexing

   [junit4] Completed [149/438] on J0 in 0.55s, 34 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestRecyclingIntBlockAllocator

   [junit4] Completed [150/438] on J0 in 0.19s, 3 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestSimpleExplanationsWithFillerDocs

   [junit4] IGNOR/A 0.01s J0 | TestSimpleExplanationsWithFillerDocs.testMA2

   [junit4]    > Assumption #1: got: <[test is not viable with empty filler docs, null]>, expected: each not null

   [junit4] IGNOR/A 0.00s J0 | TestSimpleExplanationsWithFillerDocs.testMA1

   [junit4]    > Assumption #1: got: <[test is not viable with empty filler docs, null]>, expected: each not null

   [junit4] Completed [151/438] on J0 in 7.50s, 70 tests, 2 skipped

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestParallelCompositeReader

   [junit4] Completed [152/438] on J0 in 0.78s, 15 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestPrefixQuery

   [junit4] Completed [153/438] on J0 in 10.42s, 3 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.spans.TestSpanMultiTermQueryWrapper

   [junit4] Completed [154/438] on J0 in 0.08s, 9 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestTermRangeQuery

   [junit4] Completed [155/438] on J0 in 0.22s, 7 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestTermVectorsReader

   [junit4] Completed [156/438] on J0 in 0.34s, 13 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestSPIClassIterator

   [junit4] Completed [157/438] on J0 in 0.03s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.store.TestDirectory

   [junit4] Completed [158/438] on J0 in 0.53s, 3 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestMatchNoDocsQuery

   [junit4] Completed [159/438] on J0 in 0.02s, 3 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.similarities.TestClassicSimilarity

   [junit4] Completed [160/438] on J0 in 0.11s, 11 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestIndexReaderClose

   [junit4] Completed [161/438] on J0 in 1.02s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestIndexWriter

   [junit4] Completed [162/438] on J0 in 9.86s, 71 tests

   [junit4] 

   [junit4] HEARTBEAT J1 PID(9178@localhost): 2016-07-31T05:49:09, stalled for  432s at: TestDuelingCodecsAtNight.testBigEquals

   [junit4] HEARTBEAT J2 PID(9177@localhost): 2016-07-31T05:49:21, stalled for  132s at: TestDocumentsWriterStallControl.testAccquireReleaseRace

   [junit4] HEARTBEAT J1 PID(9178@localhost): 2016-07-31T05:50:09, stalled for  492s at: TestDuelingCodecsAtNight.testBigEquals

   [junit4] HEARTBEAT J2 PID(9177@localhost): 2016-07-31T05:50:21, stalled for  192s at: TestDocumentsWriterStallControl.testAccquireReleaseRace

   [junit4] Suite: org.apache.lucene.codecs.perfield.TestPerFieldDocValuesFormat

   [junit4] Completed [163/438] on J0 in 83.53s, 96 tests

   [junit4] 

   [junit4] HEARTBEAT J1 PID(9178@localhost): 2016-07-31T05:51:09, stalled for  552s at: TestDuelingCodecsAtNight.testBigEquals

   [junit4] Suite: org.apache.lucene.index.TestIndexWriterReader

   [junit4] Completed [164/438] on J0 in 42.46s, 23 tests

   [junit4] 

   [junit4] HEARTBEAT J2 PID(9177@localhost): 2016-07-31T05:51:21, stalled for  252s at: TestDocumentsWriterStallControl.testAccquireReleaseRace

   [junit4] HEARTBEAT J1 PID(9178@localhost): 2016-07-31T05:52:09, stalled for  612s at: TestDuelingCodecsAtNight.testBigEquals

   [junit4] HEARTBEAT J2 PID(9177@localhost): 2016-07-31T05:52:21, stalled for  312s at: TestDocumentsWriterStallControl.testAccquireReleaseRace

   [junit4] HEARTBEAT J0 PID(9183@localhost): 2016-07-31T05:52:45, stalled for 60.1s at: TestFSTs.testBigSet

   [junit4] Suite: org.apache.lucene.util.fst.TestFSTs

   [junit4] Completed [165/438] on J0 in 96.63s, 20 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestNumericRangeQuery64

   [junit4] Completed [166/438] on J0 in 11.25s, 29 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestIndexWriterWithThreads

   [junit4] IGNOR/A 0.01s J0 | TestIndexWriterWithThreads.testOpenTwoIndexWritersOnDifferentThreads

   [junit4]    > Assumption #1: aborting test: timeout obtaining lock

   [junit4] Completed [167/438] on J0 in 9.42s, 12 tests, 1 skipped

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestDuelingCodecsAtNight

   [junit4] Completed [168/438] on J1 in 690.93s, 3 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestLSBRadixSorter

   [junit4] Completed [169/438] on J1 in 0.08s, 6 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestAddIndexes

   [junit4] Completed [170/438] on J0 in 6.36s, 25 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestMinShouldMatch2

   [junit4] Completed [171/438] on J1 in 3.64s, 6 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestNewestSegment

   [junit4] Completed [172/438] on J1 in 0.02s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestVirtualMethod

   [junit4] Completed [173/438] on J1 in 0.07s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestIndexWriterConfig

   [junit4] Completed [174/438] on J1 in 0.11s, 8 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.codecs.compressing.TestHighCompressionMode

   [junit4] Completed [175/438] on J1 in 3.69s, 7 tests

   [junit4] 

   [junit4] HEARTBEAT J2 PID(9177@localhost): 2016-07-31T05:53:21, stalled for  372s at: TestDocumentsWriterStallControl.testAccquireReleaseRace

   [junit4] Suite: org.apache.lucene.search.TestShardSearching

   [junit4] Completed [176/438] on J0 in 16.55s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestDeletionPolicy

   [junit4] Completed [177/438] on J0 in 5.00s, 6 tests

   [junit4] 

   [junit4] HEARTBEAT J2 PID(9177@localhost): 2016-07-31T05:54:21, stalled for  432s at: TestDocumentsWriterStallControl.testAccquireReleaseRace

   [junit4] HEARTBEAT J1 PID(9178@localhost): 2016-07-31T05:54:33, stalled for 63.9s at: TestMixedDocValuesUpdates.testTonsOfUpdates

   [junit4] HEARTBEAT J0 PID(9183@localhost): 2016-07-31T05:54:45, stalled for 69.0s at: TestNRTThreads.testNRTThreads

   [junit4] Suite: org.apache.lucene.index.TestMixedDocValuesUpdates

   [junit4] Completed [178/438] on J1 in 115.25s, 4 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestConstantScoreQuery

   [junit4] Completed [179/438] on J1 in 0.06s, 5 tests

   [junit4] 

   [junit4] HEARTBEAT J2 PID(9177@localhost): 2016-07-31T05:55:21, stalled for  492s at: TestDocumentsWriterStallControl.testAccquireReleaseRace

   [junit4] HEARTBEAT J0 PID(9183@localhost): 2016-07-31T05:55:45, stalled for  129s at: TestNRTThreads.testNRTThreads

   [junit4] HEARTBEAT J2 PID(9177@localhost): 2016-07-31T05:56:21, stalled for  552s at: TestDocumentsWriterStallControl.testAccquireReleaseRace

   [junit4] HEARTBEAT J1 PID(9178@localhost): 2016-07-31T05:56:33, stalled for 67.4s at: TestIndexSorting.testRandom3

   [junit4] Suite: org.apache.lucene.index.TestDocumentsWriterStallControl

   [junit4] Completed [180/438] on J2 in 569.89s, 3 tests

   [junit4] 

   [junit4] HEARTBEAT J0 PID(9183@localhost): 2016-07-31T05:56:46, stalled for  189s at: TestNRTThreads.testNRTThreads

   [junit4] Suite: org.apache.lucene.index.TestDuelingCodecs

   [junit4] Completed [181/438] on J2 in 23.70s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestBoolean2

   [junit4] Completed [182/438] on J2 in 3.28s, 11 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestStressAdvance

   [junit4] Completed [183/438] on J2 in 13.12s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestNorms

   [junit4] Completed [184/438] on J2 in 5.43s, 2 tests

   [junit4] 

   [junit4] HEARTBEAT J1 PID(9178@localhost): 2016-07-31T05:57:34, stalled for  127s at: TestIndexSorting.testRandom3

   [junit4] Suite: org.apache.lucene.analysis.standard.TestStandardAnalyzer

   [junit4] Completed [185/438] on J2 in 13.89s, 34 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestIndexWriterCommit

   [junit4] Completed [186/438] on J2 in 2.13s, 14 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestTimeLimitingCollector

   [junit4] Completed [187/438] on J2 in 1.91s, 7 tests

   [junit4] 

   [junit4] HEARTBEAT J0 PID(9183@localhost): 2016-07-31T05:57:46, stalled for  249s at: TestNRTThreads.testNRTThreads

   [junit4] Suite: org.apache.lucene.search.TestSearchWithThreads

   [junit4] Completed [188/438] on J2 in 6.42s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestSearchAfter

   [junit4] Completed [189/438] on J2 in 1.45s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestConcurrentMergeScheduler

   [junit4] Completed [190/438] on J2 in 8.59s, 16 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.automaton.TestMinimize

   [junit4] Completed [191/438] on J2 in 1.99s, 3 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestIndexSorting

   [junit4] Completed [192/438] on J1 in 189.82s, 27 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestByteBlockPool

   [junit4] Completed [193/438] on J1 in 0.14s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.Test2BSortedDocValuesOrds

   [junit4] IGNOR/A 0.00s J1 | Test2BSortedDocValuesOrds.test2BOrds

   [junit4]    > Assumption #1: 'monster' test group is disabled (@Monster(value=Takes ~ 6 hours if the heap is 5gb))

   [junit4] Completed [194/438] on J1 in 0.01s, 1 test, 1 skipped

   [junit4] 

   [junit4] Suite: org.apache.lucene.TestAssertions

   [junit4] Completed [195/438] on J1 in 0.03s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestRollingUpdates

   [junit4] Completed [196/438] on J2 in 22.79s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestSnapshotDeletionPolicy

   [junit4] Completed [197/438] on J2 in 1.42s, 7 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestIndexWriterUnicode

   [junit4] Completed [198/438] on J2 in 4.69s, 5 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestIndexWriterForceMerge

   [junit4] Completed [199/438] on J2 in 4.62s, 4 tests

   [junit4] 

   [junit4] HEARTBEAT J0 PID(9183@localhost): 2016-07-31T05:58:46, stalled for  309s at: TestNRTThreads.testNRTThreads

   [junit4] Suite: org.apache.lucene.codecs.perfield.TestPerFieldPostingsFormat

   [junit4] IGNOR/A 0.00s J2 | TestPerFieldPostingsFormat.testPostingsEnumReuse

   [junit4]    > Assumption #1: The MockRandom PF randomizes content on the fly, so we can't check it

   [junit4] IGNOR/A 0.01s J2 | TestPerFieldPostingsFormat.testMergeStability

   [junit4]    > Assumption #1: The MockRandom PF randomizes content on the fly, so we can't check it

   [junit4] Completed [200/438] on J2 in 22.99s, 24 tests, 2 skipped

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestArrayUtil

   [junit4] Completed [201/438] on J2 in 4.27s, 14 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestNRTThreads

   [junit4] Completed [202/438] on J0 in 334.79s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestWeakIdentityMap

   [junit4] Completed [203/438] on J0 in 15.78s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestDirectoryReader

   [junit4] Completed [204/438] on J0 in 0.89s, 24 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestSegmentMerger

   [junit4] Completed [205/438] on J0 in 0.47s, 3 tests

   [junit4] 

   [junit4] HEARTBEAT J1 PID(9178@localhost): 2016-07-31T05:59:58, stalled for 63.2s at: TestIndexWriterExceptions.testTooManyTokens

   [junit4] HEARTBEAT J2 PID(9177@localhost): 2016-07-31T06:00:10, stalled for 65.9s at: TestTermsEnum.testVaryingTermsPerSegment

   [junit4] HEARTBEAT J0 PID(9183@localhost): 2016-07-31T06:00:34, stalled for 65.0s at: TestBagOfPositions.test

   [junit4] HEARTBEAT J1 PID(9178@localhost): 2016-07-31T06:00:58, stalled for  123s at: TestIndexWriterExceptions.testTooManyTokens

   [junit4] HEARTBEAT J2 PID(9177@localhost): 2016-07-31T06:01:10, stalled for  126s at: TestTermsEnum.testVaryingTermsPerSegment

   [junit4] HEARTBEAT J0 PID(9183@localhost): 2016-07-31T06:01:34, stalled for  125s at: TestBagOfPositions.test

   [junit4] Suite: org.apache.lucene.index.TestTermsEnum

   [junit4] Completed [206/438] on J2 in 150.34s, 11 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestOmitNorms

   [junit4] Completed [207/438] on J2 in 0.63s, 5 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestComplexExplanations

   [junit4] Completed [208/438] on J2 in 0.72s, 21 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.store.TestNRTCachingDirectory

   [junit4] IGNOR/A 0.00s J2 | TestNRTCachingDirectory.testFsyncDoesntCreateNewFiles

   [junit4]    > Assumption #1: test only works for FSDirectory subclasses

   [junit4] IGNOR/A 0.01s J2 | TestNRTCachingDirectory.testPendingDeletions

   [junit4]    > Assumption #1: we can only install VirusCheckingFS on an FSDirectory

   [junit4] Completed [209/438] on J2 in 2.75s, 46 tests, 2 skipped

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestIndexWriterNRTIsCurrent

   [junit4] Completed [210/438] on J2 in 6.10s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestFieldCacheRewriteMethod

   [junit4] Completed [211/438] on J2 in 4.15s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestMultiThreadTermVectors

   [junit4] Completed [212/438] on J2 in 1.10s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestSegmentTermDocs

   [junit4] Completed [213/438] on J2 in 0.42s, 4 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestIndexWriterExceptions

   [junit4] Completed [214/438] on J1 in 208.21s, 36 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestRadixSelector

   [junit4] Completed [215/438] on J1 in 0.38s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.codecs.perfield.TestPerFieldPostingsFormat2

   [junit4] Completed [216/438] on J2 in 2.71s, 5 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.codecs.lucene50.TestLucene50FieldInfoFormat

   [junit4] Completed [217/438] on J1 in 0.78s, 11 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestDocValuesScoring

   [junit4] Completed [218/438] on J1 in 0.06s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.codecs.compressing.TestFastCompressionMode

   [junit4] Completed [219/438] on J1 in 1.99s, 11 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.automaton.TestDeterminism

   [junit4] Completed [220/438] on J1 in 0.79s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.store.TestSimpleFSDirectory

   [junit4] Completed [221/438] on J1 in 4.20s, 44 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestSegmentReader

   [junit4] Completed [222/438] on J1 in 1.52s, 7 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestMultiDocValues

   [junit4] Completed [223/438] on J2 in 11.10s, 8 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.analysis.TestToken

   [junit4] Completed [224/438] on J2 in 0.01s, 5 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestFieldsReader

   [junit4] Completed [225/438] on J2 in 0.22s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestCustomSearcherSort

   [junit4] Completed [226/438] on J2 in 1.73s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestTermsEnum2

   [junit4] Completed [227/438] on J2 in 3.05s, 4 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestMixedCodecs

   [junit4] Completed [228/438] on J2 in 3.48s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestTransactionRollback

   [junit4] Completed [229/438] on J2 in 0.39s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.automaton.TestUTF32ToUTF8

   [junit4] Completed [230/438] on J2 in 7.37s, 6 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestFixedBitSet

   [junit4] Completed [231/438] on J1 in 19.71s, 19 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestReadOnlyIndex

   [junit4] Completed [232/438] on J1 in 0.06s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestSameTokenSamePosition

   [junit4] Completed [233/438] on J1 in 0.05s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.store.TestLockFactory

   [junit4] Completed [234/438] on J1 in 0.06s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestMultiset

   [junit4] Completed [235/438] on J1 in 0.02s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestStressIndexing2

   [junit4] Completed [236/438] on J2 in 1.48s, 3 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.similarities.TestSimilarityBase

   [junit4] Completed [237/438] on J1 in 1.32s, 32 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestMultiPhraseQuery

   [junit4] Completed [238/438] on J2 in 0.27s, 17 tests, 1 skipped

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestThreadedForceMerge

   [junit4] Completed [239/438] on J1 in 0.83s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestLazyProxSkipping

   [junit4] Completed [240/438] on J2 in 0.89s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestDocsAndPositions

   [junit4] Completed [241/438] on J2 in 1.20s, 6 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.TestMergeSchedulerExternal

   [junit4] Completed [242/438] on J2 in 0.64s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestBytesRefHash

   [junit4] Completed [243/438] on J2 in 3.04s, 8 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestStressNRT

   [junit4] Completed [244/438] on J2 in 0.22s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestIndexInput

   [junit4] Completed [245/438] on J2 in 1.01s, 2 tests

   [junit4] 

   [junit4] HEARTBEAT J0 PID(9183@localhost): 2016-07-31T06:02:34, stalled for  185s at: TestBagOfPositions.test

   [junit4] Suite: org.apache.lucene.search.TestSloppyPhraseQuery2

   [junit4] Completed [246/438] on J2 in 13.76s, 9 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestDisjunctionMaxQuery

   [junit4] Completed [247/438] on J2 in 0.50s, 11 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestForceMergeForever

   [junit4] Completed [248/438] on J2 in 1.37s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.spans.TestNearSpansOrdered

   [junit4] Completed [249/438] on J2 in 0.46s, 19 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestSameScoresWithThreads

   [junit4] Completed [250/438] on J2 in 0.77s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestMultiFields

   [junit4] Completed [251/438] on J2 in 1.86s, 3 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestDocumentWriter

   [junit4] Completed [252/438] on J2 in 0.08s, 6 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestCrashCausesCorruptIndex

   [junit4] Completed [253/438] on J2 in 0.05s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestMultiTermConstantScore

   [junit4] Completed [254/438] on J2 in 3.68s, 8 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestLiveFieldValues

   [junit4] Completed [255/438] on J2 in 0.73s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.spans.TestSpanSearchEquivalence

   [junit4] Completed [256/438] on J2 in 31.66s, 33 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestFuzzyQuery

   [junit4] Completed [257/438] on J2 in 0.21s, 10 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.TestExternalCodecs

   [junit4] Completed [258/438] on J2 in 4.62s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestNoDeletionPolicy

   [junit4] Completed [259/438] on J2 in 0.08s, 4 tests

   [junit4] 

   [junit4] HEARTBEAT J0 PID(9183@localhost): 2016-07-31T06:03:34, stalled for  245s at: TestBagOfPositions.test

   [junit4] HEARTBEAT J1 PID(9178@localhost): 2016-07-31T06:03:34, stalled for 69.5s at: TestNumericDocValuesUpdates.testTonsOfUpdates

   [junit4] Suite: org.apache.lucene.search.TestRegexpQuery

   [junit4] Completed [260/438] on J2 in 0.16s, 7 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestPerSegmentDeletes

   [junit4] Completed [261/438] on J2 in 0.03s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestIntBlockPool

   [junit4] Completed [262/438] on J2 in 0.44s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestPrefixInBooleanQuery

   [junit4] Completed [263/438] on J2 in 2.02s, 4 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestSegmentTermEnum

   [junit4] Completed [264/438] on J2 in 0.08s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestUniqueTermCount

   [junit4] Completed [265/438] on J2 in 0.08s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestBooleanQuery

   [junit4] Completed [266/438] on J2 in 0.42s, 22 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestPrefixCodedTerms

   [junit4] Completed [267/438] on J2 in 0.35s, 5 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestAllFilesHaveCodecHeader

   [junit4] Completed [268/438] on J2 in 0.24s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestAutomatonQuery

   [junit4] Completed [269/438] on J2 in 1.30s, 8 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.document.TestDocument

   [junit4] Completed [270/438] on J2 in 0.09s, 12 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.store.TestFileSwitchDirectory

   [junit4] IGNOR/A 0.00s J2 | TestFileSwitchDirectory.testFsyncDoesntCreateNewFiles

   [junit4]    > Assumption #1: test only works for FSDirectory subclasses

   [junit4] IGNOR/A 0.00s J2 | TestFileSwitchDirectory.testPendingDeletions

   [junit4]    > Assumption #1: we can only install VirusCheckingFS on an FSDirectory

   [junit4] Completed [271/438] on J2 in 1.62s, 45 tests, 2 skipped

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestSentinelIntSet

   [junit4] Completed [272/438] on J2 in 0.13s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestWildcardRandom

   [junit4] Completed [273/438] on J2 in 0.27s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestPrefixRandom

   [junit4] Completed [274/438] on J2 in 1.26s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestRegexpRandom

   [junit4] Completed [275/438] on J2 in 0.06s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestCheckIndex

   [junit4] Completed [276/438] on J2 in 0.48s, 5 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestPayloadsOnVectors

   [junit4] Completed [277/438] on J2 in 0.03s, 3 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.TestSearchForDuplicates

   [junit4] Completed [278/438] on J2 in 0.15s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestIndexFileDeleter

   [junit4] Completed [279/438] on J2 in 1.03s, 10 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestParallelTermEnum

   [junit4] Completed [280/438] on J2 in 0.01s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestBinaryTerms

   [junit4] Completed [281/438] on J2 in 0.05s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestSubScorerFreqs

   [junit4] Completed [282/438] on J2 in 0.04s, 3 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestConjunctions

   [junit4] Completed [283/438] on J2 in 0.01s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestMultiTermQueryRewrites

   [junit4] Completed [284/438] on J2 in 0.02s, 3 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.document.TestField

   [junit4] Completed [285/438] on J2 in 0.05s, 27 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.spans.TestSpanExplanationsOfNonMatches

   [junit4] Completed [286/438] on J2 in 0.03s, 30 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestRollingBuffer

   [junit4] Completed [287/438] on J2 in 0.15s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.analysis.TestNumericTokenStream

   [junit4] Completed [288/438] on J2 in 0.02s, 6 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestFilterLeafReader

   [junit4] Completed [289/438] on J2 in 0.04s, 3 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.TestSearch

   [junit4] Completed [290/438] on J2 in 0.04s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestFilterIterator

   [junit4] Completed [291/438] on J2 in 0.01s, 8 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.document.TestDateTools

   [junit4] Completed [292/438] on J2 in 0.02s, 5 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestCharsRef

   [junit4] Completed [293/438] on J2 in 0.03s, 5 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestBagOfPositions

   [junit4] Completed [294/438] on J0 in 265.09s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestBooleanScorer

   [junit4] Completed [295/438] on J0 in 7.95s, 5 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestRamUsageEstimator

   [junit4] IGNOR/A 0.00s J0 | TestRamUsageEstimator.testPrintValues

   [junit4]    > Assumption #1: Specify -Dtests.verbose=true to print constants of RamUsageEstimator.

   [junit4] Completed [296/438] on J0 in 0.01s, 5 tests, 1 skipped

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestIsCurrent

   [junit4] Completed [297/438] on J0 in 0.07s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestNoMergePolicy

   [junit4] Completed [298/438] on J0 in 0.02s, 4 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestNoMergeScheduler

   [junit4] Completed [299/438] on J0 in 0.01s, 3 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestNamedSPILoader

   [junit4] Completed [300/438] on J0 in 0.01s, 3 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestIndexWriterLockRelease

   [junit4] Completed [301/438] on J0 in 0.01s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestPhrasePrefixQuery

   [junit4] Completed [302/438] on J0 in 0.02s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.analysis.TestCharFilter

   [junit4] Completed [303/438] on J0 in 0.01s, 4 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestRollback

   [junit4] Completed [304/438] on J0 in 0.01s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestVersion

   [junit4] Completed [305/438] on J0 in 0.03s, 11 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestNumericDocValuesUpdates

   [junit4] Completed [306/438] on J1 in 121.51s, 31 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.packed.TestDirectPacked

   [junit4] Completed [307/438] on J1 in 9.44s, 4 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestTermVectors

   [junit4] Completed [308/438] on J1 in 0.43s, 3 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestMaxPosition

   [junit4] Completed [309/438] on J1 in 0.04s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.spans.TestSpanExplanations

   [junit4] Completed [310/438] on J1 in 0.45s, 30 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestUsageTrackingFilterCachingPolicy

   [junit4] Completed [311/438] on J1 in 0.03s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestAttributeSource

   [junit4] Completed [312/438] on J1 in 0.07s, 7 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestNumericUtils

   [junit4] Completed [313/438] on J1 in 1.41s, 24 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestByteSlices

   [junit4] Completed [314/438] on J1 in 5.14s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestTopDocsCollector

   [junit4] Completed [315/438] on J1 in 0.04s, 8 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestParallelLeafReader

   [junit4] Completed [316/438] on J1 in 0.32s, 9 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.bkd.TestMutablePointsReaderUtils

   [junit4] Completed [317/438] on J1 in 0.94s, 3 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestSimpleSearchEquivalence

   [junit4] Completed [318/438] on J1 in 9.08s, 17 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestRegexpRandom2

   [junit4] Completed [319/438] on J1 in 4.36s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestFixedLengthBytesRefArray

   [junit4] Completed [320/438] on J1 in 0.15s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.document.TestBinaryDocument

   [junit4] Completed [321/438] on J1 in 0.02s, 2 tests

   [junit4] 

   [junit4] HEARTBEAT J2 PID(9177@localhost): 2016-07-31T06:04:58, stalled for 66.7s at: Test2BPostings.test

   [junit4] Suite: org.apache.lucene.index.TestDocumentsWriterDeleteQueue

   [junit4] Completed [322/438] on J1 in 0.36s, 5 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestStringHelper

   [junit4] Completed [323/438] on J1 in 0.04s, 7 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestSortRescorer

   [junit4] Completed [324/438] on J1 in 0.73s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestPostingsOffsets

   [junit4] Completed [325/438] on J1 in 1.99s, 13 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestScorerPerf

   [junit4] Completed [326/438] on J1 in 8.86s, 1 test

   [junit4] 

   [junit4] HEARTBEAT J0 PID(9183@localhost): 2016-07-31T06:05:10, stalled for 65.4s at: Test4GBStoredFields.test

   [junit4] Suite: org.apache.lucene.search.spans.TestSpans

   [junit4] Completed [327/438] on J1 in 0.86s, 26 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestTopDocsMerge

   [junit4] Completed [328/438] on J1 in 36.75s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestTransactions

   [junit4] Completed [329/438] on J1 in 3.27s, 1 test

   [junit4] 

   [junit4] HEARTBEAT J2 PID(9177@localhost): 2016-07-31T06:05:58, stalled for  127s at: Test2BPostings.test

   [junit4] HEARTBEAT J0 PID(9183@localhost): 2016-07-31T06:06:10, stalled for  125s at: Test4GBStoredFields.test

   [junit4] Suite: org.apache.lucene.util.TestTimSorterWorstCase

   [junit4] Completed [330/438] on J1 in 26.31s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.store.TestNIOFSDirectory

   [junit4] Completed [331/438] on J1 in 3.97s, 44 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestPositionIncrement

   [junit4] Completed [332/438] on J1 in 0.02s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.spans.TestFieldMaskingSpanQuery

   [junit4] IGNOR/A 0.00s J1 | TestFieldMaskingSpanQuery.testSimple2

   [junit4]    > Assumption #1: Broken scoring: LUCENE-3723

   [junit4] IGNOR/A 0.00s J1 | TestFieldMaskingSpanQuery.testSpans2

   [junit4]    > Assumption #1: Broken scoring: LUCENE-3723

   [junit4] Completed [333/438] on J1 in 0.22s, 11 tests, 2 skipped

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestFlushByRamOrCountsPolicy

   [junit4] Completed [334/438] on J1 in 7.18s, 5 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.MultiCollectorTest

   [junit4] Completed [335/438] on J1 in 0.01s, 4 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestDocBoost

   [junit4] Completed [336/438] on J1 in 0.01s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestNGramPhraseQuery

   [junit4] Completed [337/438] on J1 in 0.01s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestScoreCachingWrappingScorer

   [junit4] Completed [338/438] on J1 in 0.02s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestIntsRef

   [junit4] Completed [339/438] on J1 in 0.01s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.analysis.tokenattributes.TestSimpleAttributeImpl

   [junit4] Completed [340/438] on J1 in 0.01s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestTermdocPerf

   [junit4] Completed [341/438] on J1 in 0.01s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestIndexCommit

   [junit4] Completed [342/438] on J1 in 0.03s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestTerm

   [junit4] Completed [343/438] on J1 in 0.03s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.Test2BPostingsBytes

   [junit4] IGNOR/A 0.00s J1 | Test2BPostingsBytes.test

   [junit4]    > Assumption #1: 'monster' test group is disabled (@Monster(value=takes ~20GB-30GB of space and 10 minutes))

   [junit4] Completed [344/438] on J1 in 0.00s, 1 test, 1 skipped

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestPositiveScoresOnlyCollector

   [junit4] Completed [345/438] on J1 in 0.02s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.Test2BTerms

   [junit4] IGNOR/A 0.00s J1 | Test2BTerms.test2BTerms

   [junit4]    > Assumption #1: 'monster' test group is disabled (@Monster(value=very slow, use 5g minimum heap))

   [junit4] Completed [346/438] on J1 in 0.01s, 1 test, 1 skipped

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.Test2BNumericDocValues

   [junit4] IGNOR/A 0.00s J1 | Test2BNumericDocValues.testNumerics

   [junit4]    > Assumption #1: 'monster' test group is disabled (@Monster(value=takes ~ 2 hours if the heap is 5gb))

   [junit4] Completed [347/438] on J1 in 0.00s, 1 test, 1 skipped

   [junit4] 

   [junit4] Suite: org.apache.lucene.analysis.TestCharArrayMap

   [junit4] Completed [349/438] on J1 in 0.08s, 4 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.analysis.TestCharArraySet

   [junit4] Completed [350/438] on J1 in 0.02s, 15 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.analysis.TestCharacterUtils

   [junit4] Completed [351/438] on J1 in 0.02s, 4 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.analysis.TestReusableStringReader

   [junit4] Completed [352/438] on J1 in 0.01s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.analysis.TestStopFilter

   [junit4] Completed [353/438] on J1 in 0.01s, 4 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.analysis.tokenattributes.TestBytesRefAttImpl

   [junit4] Completed [354/438] on J1 in 0.01s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.analysis.tokenattributes.TestPackedTokenAttributeImpl

   [junit4] Completed [355/438] on J1 in 0.01s, 4 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.codecs.TestCodecLoadingDeadlock

   [junit4] Completed [356/438] on J1 in 0.32s, 1 test

   [junit4] 

   [junit4] HEARTBEAT J2 PID(9177@localhost): 2016-07-31T06:06:58, stalled for  187s at: Test2BPostings.test

   [junit4] HEARTBEAT J0 PID(9183@localhost): 2016-07-31T06:07:10, stalled for  185s at: Test4GBStoredFields.test

   [junit4] Suite: org.apache.lucene.index.Test4GBStoredFields

   [junit4] Completed [357/438] on J0 in 211.53s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.codecs.compressing.TestGrowableByteArrayDataOutput

   [junit4] Completed [358/438] on J1 in 67.76s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.codecs.lucene50.TestBlockPostingsFormat2

   [junit4] Completed [359/438] on J1 in 3.19s, 4 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.codecs.lucene50.TestBlockPostingsFormat3

   [junit4] Completed [360/438] on J1 in 8.23s, 1 test

   [junit4] 

   [junit4] HEARTBEAT J2 PID(9177@localhost): 2016-07-31T06:07:58, stalled for  247s at: Test2BPostings.test

   [junit4] Suite: org.apache.lucene.codecs.lucene50.TestBlockPostingsFormat

   [junit4] Completed [361/438] on J0 in 23.19s, 26 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.codecs.lucene50.TestLucene50StoredFieldsFormat

   [junit4] Completed [362/438] on J1 in 14.11s, 17 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.codecs.lucene50.TestLucene50StoredFieldsFormatHighCompression

   [junit4] Completed [363/438] on J0 in 18.70s, 19 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.codecs.lucene50.TestLucene50TermVectorsFormat

   [junit4] Completed [364/438] on J1 in 21.76s, 17 tests

   [junit4] 

   [junit4] HEARTBEAT J2 PID(9177@localhost): 2016-07-31T06:08:58, stalled for  307s at: Test2BPostings.test

   [junit4] HEARTBEAT J0 PID(9183@localhost): 2016-07-31T06:09:58, stalled for 67.4s at: TestLucene53NormsFormat.testNCommonBig

   [junit4] HEARTBEAT J2 PID(9177@localhost): 2016-07-31T06:09:58, stalled for  367s at: Test2BPostings.test

   [junit4] Suite: org.apache.lucene.codecs.lucene53.TestLucene53NormsFormat

   [junit4] IGNOR/A 0.00s J0 | TestLucene53NormsFormat.testMergeStability

   [junit4]    > Assumption #1: The MockRandom PF randomizes content on the fly, so we can't check it

   [junit4] Completed [365/438] on J0 in 108.33s, 17 tests, 1 skipped

   [junit4] 

   [junit4] Suite: org.apache.lucene.codecs.lucene60.TestLucene60PointsFormat

   [junit4] IGNOR/A 0.00s J0 | TestLucene60PointsFormat.testMergeStability

   [junit4]    > Assumption #1: merge is not stable

   [junit4] Completed [366/438] on J0 in 13.49s, 17 tests, 1 skipped

   [junit4] 

   [junit4] Suite: org.apache.lucene.codecs.lucene62.TestLucene62SegmentInfoFormat

   [junit4] Completed [367/438] on J0 in 0.26s, 16 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.document.TestFieldType

   [junit4] Completed [368/438] on J0 in 0.01s, 3 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.geo.TestGeoEncodingUtils

   [junit4] Completed [369/438] on J0 in 0.50s, 3 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.geo.TestGeoUtils

   [junit4] Completed [370/438] on J0 in 18.53s, 6 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.geo.TestPolygon

   [junit4] Completed [371/438] on J0 in 0.03s, 15 tests

   [junit4] 

   [junit4] HEARTBEAT J2 PID(9177@localhost): 2016-07-31T06:10:58, stalled for  427s at: Test2BPostings.test

   [junit4] Suite: org.apache.lucene.geo.TestPolygon2D

   [junit4] Completed [372/438] on J0 in 22.72s, 10 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.Test2BPoints

   [junit4] IGNOR/A 0.01s J0 | Test2BPoints.test1D

   [junit4]    > Assumption #1: 'monster' test group is disabled (@Monster(value=takes at least 4 hours and consumes many GB of temp disk space))

   [junit4] IGNOR/A 0.00s J0 | Test2BPoints.test2D

   [junit4]    > Assumption #1: 'monster' test group is disabled (@Monster(value=takes at least 4 hours and consumes many GB of temp disk space))

   [junit4] Completed [373/438] on J0 in 0.01s, 2 tests, 2 skipped

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.Test2BSortedDocValuesFixedSorted

   [junit4] IGNOR/A 0.00s J0 | Test2BSortedDocValuesFixedSorted.testFixedSorted

   [junit4]    > Assumption #1: 'monster' test group is disabled (@Monster(value=Takes ~ 6 hours if the heap is 5gb))

   [junit4] Completed [374/438] on J0 in 0.01s, 1 test, 1 skipped

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.Test2BPostings

   [junit4] Completed [375/438] on J2 in 435.89s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestAllFilesCheckIndexHeader

   [junit4] Completed [376/438] on J0 in 4.59s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestAllFilesDetectTruncation

   [junit4] Completed [377/438] on J2 in 8.70s, 1 test

   [junit4] 

   [junit4] HEARTBEAT J1 PID(9178@localhost): 2016-07-31T06:12:10, stalled for 67.4s at: TestLucene54DocValuesFormat.testSortedSetVariableLengthBigVsStoredFields

   [junit4] HEARTBEAT J0 PID(9183@localhost): 2016-07-31T06:12:10, stalled for 61.1s at: TestBinaryDocValuesUpdates.testTonsOfUpdates

   [junit4] Suite: org.apache.lucene.codecs.lucene54.TestLucene54DocValuesFormat

   [junit4] Completed [378/438] on J1 in 246.98s, 109 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestDocIDMerger

   [junit4] Completed [379/438] on J1 in 0.04s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestDocValues

   [junit4] Completed [380/438] on J1 in 0.09s, 7 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestExceedMaxTermLength

   [junit4] Completed [381/438] on J1 in 0.01s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestFieldReuse

   [junit4] Completed [3

[...truncated too long message...]

-6.x/solr/core/test-lib/apacheds-interceptors-schema-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-subtree-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-trigger-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-jdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-jdbm1-2.0.0-M2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-kerberos-codec-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-ldif-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-mavibot-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-protocol-kerberos-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-protocol-ldap-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-protocol-shared-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-xdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/api-all-1.0.0-M20.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/bcprov-jdk15-1.45.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/cglib-nodep-2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/commons-collections-3.2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/commons-math3-3.4.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/easymock-3.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/ehcache-core-2.4.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/hadoop-common-2.7.2-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/hadoop-hdfs-2.7.2-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/hadoop-minikdc-2.7.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/htrace-core-3.2.0-incubating.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jackson-annotations-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jackson-core-asl-1.9.13.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jackson-databind-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jackson-mapper-asl-1.9.13.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jersey-core-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jersey-server-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jetty-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jetty-util-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/mina-core-2.0.0-M5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/mockito-core-1.9.5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/netty-all-4.0.36.Final.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/objenesis-1.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/presto-parser-0.122.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/slice-0.10.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/analysis/icu/lucene-analyzers-icu-6.2.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/contrib/solr-analysis-extras/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/queryparser/classes/test:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/contrib/analysis-extras/lib/icu4j-56.1.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-launcher.jar:/x1/jenkins/.ant/lib/ivy-2.3.0.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-antlr.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-regexp.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-resolver.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-net.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jmf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-oro.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-xalan2.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-logging.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jsch.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit4.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-javamail.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jai.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jdepend.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bsf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-swing.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-netrexx.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bcel.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-testutil.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-log4j.jar:/x1/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.8/lib/tools.jar:/x1/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.3.4.jar com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/temp/junit4-J1-20160731_081509_572.events @/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/temp/junit4-J1-20160731_081509_572.suites -stdin
   [junit4] ERROR: JVM J1 ended with an exception: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1501)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:121)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:955)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:952)
   [junit4] 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4] 	at java.lang.Thread.run(Thread.java:745)

BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:771: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:707: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/common-build.xml:536: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/common-build.xml:1443: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/common-build.xml:984: At least one slave process threw an exception, first: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.

Total time: 271 minutes 13 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
ERROR: Failed to archive {solr/build/solr-core/test/temp/junit4-J2-20160731_081509_573.events=solr/build/solr-core/test/temp/junit4-J2-20160731_081509_573.events, README.txt=README.txt, heapdumps/java_pid14710.hprof=heapdumps/java_pid14710.hprof, solr/build/solr-core/test/temp/junit4-J0-20160731_081509_571.events=solr/build/solr-core/test/temp/junit4-J0-20160731_081509_571.events, solr/build/solr-core/test/temp/junit4-J1-20160731_081509_572.events=solr/build/solr-core/test/temp/junit4-J1-20160731_081509_572.events} due to internal error; falling back to full archiving
java.lang.NullPointerException
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any





[JENKINS] Lucene-Solr-NightlyTests-6.x - Build # 131 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-6.x/131/

2 tests failed.
FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Error from server at http://127.0.0.1:58800: Error CREATEing SolrCore 'halfcollection_shard1_replica1': Unable to create core [halfcollection_shard1_replica1] Caused by: KeeperErrorCode = Session expired for /overseer/queue/qn-

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:58800: Error CREATEing SolrCore 'halfcollection_shard1_replica1': Unable to create core [halfcollection_shard1_replica1] Caused by: KeeperErrorCode = Session expired for /overseer/queue/qn-
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:590)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:259)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:248)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.makeRequest(CollectionsAPIDistributedZkTest.java:399)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testErrorHandling(CollectionsAPIDistributedZkTest.java:515)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test(CollectionsAPIDistributedZkTest.java:179)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  junit.framework.TestSuite.org.apache.solr.handler.TestReplicationHandler

Error Message:
ObjectTracker found 2 object(s) that were not released!!! [NRTCachingDirectory, NRTCachingDirectory]

Stack Trace:
java.lang.AssertionError: ObjectTracker found 2 object(s) that were not released!!! [NRTCachingDirectory, NRTCachingDirectory]
	at __randomizedtesting.SeedInfo.seed([F80A11D6CEA15901]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.junit.Assert.assertNull(Assert.java:551)
	at org.apache.solr.SolrTestCaseJ4.teardownTestCases(SolrTestCaseJ4.java:258)
	at sun.reflect.GeneratedMethodAccessor40.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:834)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
Started by timer

[EnvInject] - Loading node environment variables.

Building remotely on lucene in workspace /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x

 > git rev-parse --is-inside-work-tree # timeout=10

Fetching changes from the remote Git repository

 > git config remote.origin.url https://git-wip-us.apache.org/repos/asf/lucene-solr.git # timeout=10

Cleaning workspace

 > git rev-parse --verify HEAD # timeout=10

Resetting working tree

 > git reset --hard # timeout=10

 > git clean -fdx # timeout=10

Fetching upstream changes from https://git-wip-us.apache.org/repos/asf/lucene-solr.git

 > git --version # timeout=10

 > git -c core.askpass=true fetch --tags --progress https://git-wip-us.apache.org/repos/asf/lucene-solr.git +refs/heads/*:refs/remotes/origin/*

 > git rev-parse refs/remotes/origin/branch_6x^{commit} # timeout=10

 > git rev-parse refs/remotes/origin/origin/branch_6x^{commit} # timeout=10

Checking out Revision e3de51be2edd086c29be8fdcfca7f8a5990a640c (refs/remotes/origin/branch_6x)

 > git config core.sparsecheckout # timeout=10

 > git checkout -f e3de51be2edd086c29be8fdcfca7f8a5990a640c

 > git rev-list 6b8b34ed358287a1cadf848e14089601a3ce1671 # timeout=10

No emails were triggered.

[Lucene-Solr-NightlyTests-6.x] $ /home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/bin/ant -Dtests.multiplier=2 -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt jenkins-nightly

Buildfile: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml


jenkins-nightly:


-print-java-info:

[java-info] java version "1.8.0_74"

[java-info] Java(TM) SE Runtime Environment (1.8.0_74-b02, Oracle Corporation)

[java-info] Java HotSpot(TM) 64-Bit Server VM (25.74-b02, Oracle Corporation)

[java-info] Test args: []


clean:


clean:


clean:


ivy-availability-check:


ivy-fail:


ivy-configure:

[ivy:configure] :: Apache Ivy 2.3.0 - 20130110142753 :: http://ant.apache.org/ivy/ ::

[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/top-level-ivy-settings.xml


resolve-groovy:

[ivy:cachepath] :: resolving dependencies :: org.codehaus.groovy#groovy-all-caller;working

[ivy:cachepath] 	confs: [default]

[ivy:cachepath] 	found org.codehaus.groovy#groovy-all;2.4.6 in public

[ivy:cachepath] :: resolution report :: resolve 90ms :: artifacts dl 1ms

	---------------------------------------------------------------------

	|                  |            modules            ||   artifacts   |

	|       conf       | number| search|dwnlded|evicted|| number|dwnlded|

	---------------------------------------------------------------------

	|      default     |   1   |   0   |   0   |   0   ||   1   |   0   |

	---------------------------------------------------------------------


-test-with-heapdumps-enabled:

     [echo] Java HotSpot(TM) 64-Bit Server VM: Enabling heap dumps on OutOfMemoryError to dir '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/heapdumps'.

    [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/heapdumps


ivy-availability-check:


ivy-fail:


ivy-configure:

[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/top-level-ivy-settings.xml


resolve-groovy:


test:


ivy-availability-check:


ivy-fail:


ivy-configure:

[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/top-level-ivy-settings.xml


-clover.load:


resolve-groovy:


-init-totals:


test-core:


-clover.disable:


ivy-availability-check:


ivy-fail:


ivy-configure:

[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/top-level-ivy-settings.xml


-clover.load:


-clover.classpath:


-clover.setup:


clover:


-check-git-state:


-git-cleanroot:


-copy-git-state:

     [copy] Copying 1 file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build


git-autoclean:


resolve:


init:


compile-core:

    [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java

    [javac] Compiling 782 source files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java

    [javac] Note: Some input files use or override a deprecated API.

    [javac] Note: Recompile with -Xlint:deprecation for details.

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java/org/apache/lucene/document/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java/org/apache/lucene/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java/org/apache/lucene/util/bkd/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java/org/apache/lucene/codecs/compressing/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java/org/apache/lucene/codecs/lucene60/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java/org/apache/lucene/codecs/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java/org/apache/lucene/util/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java/org/apache/lucene/index/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java/org/apache/lucene/util/fst/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java/org/apache/lucene/analysis/tokenattributes/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java/org/apache/lucene/search/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java/org/apache/lucene/codecs/lucene62/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java/org/apache/lucene/util/packed/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java/org/apache/lucene/util/mutable/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java/org/apache/lucene/codecs/perfield/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java/org/apache/lucene/analysis/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java/org/apache/lucene/analysis/standard/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java/org/apache/lucene/store/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java/org/apache/lucene/codecs/blocktree/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java/org/apache/lucene/search/spans/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java/org/apache/lucene/codecs/lucene54/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java/org/apache/lucene/search/similarities/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java/org/apache/lucene/util/automaton/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java/org/apache/lucene/codecs/lucene53/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java/org/apache/lucene/codecs/lucene50/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java/org/apache/lucene/geo/package-info.class

     [copy] Copying 3 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java


compile-test-framework:


-check-git-state:


-git-cleanroot:


-copy-git-state:


git-autoclean:


ivy-availability-check:


ivy-fail:


ivy-configure:

[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/top-level-ivy-settings.xml


resolve:


init:


compile-lucene-core:


-check-git-state:


-git-cleanroot:


-copy-git-state:


git-autoclean:


ivy-availability-check:


ivy-fail:


ivy-configure:

[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/top-level-ivy-settings.xml


resolve:


init:


-clover.disable:


-clover.load:


-clover.classpath:


-clover.setup:


clover:


compile-core:


compile-codecs:


-check-git-state:


-git-cleanroot:


-copy-git-state:


git-autoclean:


ivy-availability-check:


ivy-fail:


ivy-configure:

[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/top-level-ivy-settings.xml


resolve:


common.init:


compile-lucene-core:


init:


-clover.disable:


-clover.load:


-clover.classpath:


-clover.setup:


clover:


compile-core:

    [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/codecs/classes/java

    [javac] Compiling 66 source files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/codecs/classes/java

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/codecs/classes/java/org/apache/lucene/codecs/bloom/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/codecs/classes/java/org/apache/lucene/codecs/memory/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/codecs/classes/java/org/apache/lucene/codecs/simpletext/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/codecs/classes/java/org/apache/lucene/codecs/blockterms/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/codecs/classes/java/org/apache/lucene/codecs/blocktreeords/package-info.class

     [copy] Copying 3 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/codecs/classes/java


-clover.disable:


-clover.load:


-clover.classpath:


-clover.setup:


clover:


common.compile-core:

    [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/test-framework/classes/java

    [javac] Compiling 185 source files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/test-framework/classes/java

    [javac] Note: Some input files use or override a deprecated API.

    [javac] Note: Recompile with -Xlint:deprecation for details.

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/test-framework/classes/java/org/apache/lucene/codecs/asserting/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/test-framework/classes/java/org/apache/lucene/codecs/ramonly/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/test-framework/classes/java/org/apache/lucene/mockfile/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/test-framework/classes/java/org/apache/lucene/codecs/cheapbastard/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/test-framework/classes/java/org/apache/lucene/codecs/mockrandom/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/test-framework/classes/java/org/apache/lucene/codecs/cranky/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/test-framework/classes/java/org/apache/lucene/codecs/compressing/dummy/package-info.class

     [copy] Copying 4 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/test-framework/classes/java


compile-core:


compile-test:

    [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/test

    [javac] Compiling 445 source files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/test

    [javac] Note: Some input files use or override a deprecated API.

    [javac] Note: Recompile with -Xlint:deprecation for details.

     [copy] Copying 3 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/test


install-junit4-taskdef:


validate:


resolve-groovy:


-init-totals:


-test:

    [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/test

[loadresource] Do not set property tests.explicitclass as its length is 0.

    [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/test/temp

    [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/.caches/test-stats/core

   [junit4] <JUnit4> says 今日は! Master seed: 24F0309E0F1E5BCB

   [junit4] Executing 438 suites with 3 JVMs.

   [junit4] 

   [junit4] Started J0 PID(30154@localhost).

   [junit4] Started J2 PID(30170@localhost).

   [junit4] Started J1 PID(30169@localhost).

   [junit4] Suite: org.apache.lucene.index.Test2BSortedDocValuesOrds

   [junit4] IGNOR/A 0.01s J1 | Test2BSortedDocValuesOrds.test2BOrds

   [junit4]    > Assumption #1: 'monster' test group is disabled (@Monster(value=Takes ~ 6 hours if the heap is 5gb))

   [junit4] Completed [1/438] on J1 in 0.19s, 1 test, 1 skipped

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestIndexSearcher

   [junit4] Completed [2/438] on J1 in 1.59s, 5 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestStringHelper

   [junit4] Completed [3/438] on J1 in 0.04s, 7 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestCrash

   [junit4] Completed [4/438] on J0 in 2.74s, 5 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestAtomicUpdate

   [junit4] Completed [5/438] on J2 in 7.88s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestUpgradeIndexMergePolicy

   [junit4] Completed [6/438] on J2 in 0.23s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestPriorityQueue

   [junit4] Completed [7/438] on J2 in 1.05s, 9 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestIndexWriterDeleteByQuery

   [junit4] Completed [8/438] on J2 in 0.36s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestIndexWriterUnicode

   [junit4] Completed [10/438] on J1 in 8.93s, 5 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestFieldCacheRewriteMethod

   [junit4] Completed [11/438] on J0 in 15.54s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestRecyclingIntBlockAllocator

   [junit4] Completed [12/438] on J0 in 0.09s, 3 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestIntBlockPool

   [junit4] Completed [13/438] on J0 in 1.48s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.automaton.TestUTF32ToUTF8

   [junit4] Completed [14/438] on J0 in 10.18s, 6 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestNRTReaderCleanup

   [junit4] Completed [15/438] on J0 in 0.07s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestNRTReaderWithThreads

   [junit4] Completed [16/438] on J0 in 1.78s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestCachingCollector

   [junit4] Completed [17/438] on J0 in 0.24s, 4 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestBytesRefArray

   [junit4] Completed [18/438] on J0 in 0.58s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.store.TestSingleInstanceLockFactory

   [junit4] Completed [19/438] on J0 in 3.34s, 7 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestDocValues

   [junit4] Completed [20/438] on J0 in 0.42s, 7 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestIndexWriterReader

   [junit4] Completed [21/438] on J2 in 58.81s, 23 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.fst.TestFSTs

   [junit4] Completed [22/438] on J1 in 64.84s, 20 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestNoMergePolicy

   [junit4] Completed [23/438] on J1 in 0.08s, 4 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.spans.TestSpanSearchEquivalence

   [junit4] Completed [24/438] on J0 in 47.71s, 33 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestSentinelIntSet

   [junit4] Completed [25/438] on J0 in 0.17s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestFieldsReader

   [junit4] Completed [26/438] on J0 in 0.72s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestTotalHitCountCollector

   [junit4] Completed [27/438] on J0 in 0.15s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestComplexExplanations

   [junit4] Completed [28/438] on J0 in 1.82s, 21 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.automaton.TestMinimize

   [junit4] Completed [29/438] on J0 in 2.47s, 3 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.FuzzyTermOnShortTermsTest

   [junit4] Completed [30/438] on J0 in 0.27s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestConjunctionDISI

   [junit4] Completed [31/438] on J0 in 10.07s, 5 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestSegmentTermDocs

   [junit4] Completed [32/438] on J0 in 0.72s, 4 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.analysis.tokenattributes.TestSimpleAttributeImpl

   [junit4] Completed [33/438] on J0 in 0.04s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestNGramPhraseQuery

   [junit4] Completed [34/438] on J0 in 0.23s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestQueryBuilder

   [junit4] Completed [35/438] on J0 in 0.08s, 21 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.store.TestHugeRamFile

   [junit4] Completed [36/438] on J0 in 1.78s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestPersistentSnapshotDeletionPolicy

   [junit4] Completed [37/438] on J0 in 1.36s, 13 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestNumericDocValuesUpdates

   [junit4] Completed [38/438] on J1 in 29.08s, 31 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestBooleanQueryVisitSubscorers

   [junit4] Completed [39/438] on J1 in 0.23s, 5 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestFixedLengthBytesRefArray

   [junit4] Completed [40/438] on J1 in 0.68s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestCharsRef

   [junit4] Completed [41/438] on J1 in 0.07s, 5 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestDocIDMerger

   [junit4] Completed [42/438] on J1 in 0.07s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.store.TestFileSwitchDirectory

   [junit4] IGNOR/A 0.01s J1 | TestFileSwitchDirectory.testPendingDeletions

   [junit4]    > Assumption #1: we can only install VirusCheckingFS on an FSDirectory

   [junit4] IGNOR/A 0.00s J1 | TestFileSwitchDirectory.testFsyncDoesntCreateNewFiles

   [junit4]    > Assumption #1: test only works for FSDirectory subclasses

   [junit4] Completed [43/438] on J1 in 2.96s, 45 tests, 2 skipped

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestVersion

   [junit4] Completed [44/438] on J1 in 0.07s, 11 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestFieldReuse

   [junit4] Completed [45/438] on J1 in 0.03s, 3 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestIndexWriterWithThreads

   [junit4] IGNOR/A 0.04s J0 | TestIndexWriterWithThreads.testOpenTwoIndexWritersOnDifferentThreads

   [junit4]    > Assumption #1: aborting test: timeout obtaining lock

   [junit4] Completed [46/438] on J0 in 9.44s, 12 tests, 1 skipped

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestAllFilesDetectTruncation

   [junit4] Completed [47/438] on J0 in 2.06s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.spans.TestBasics

   [junit4] Completed [48/438] on J1 in 7.00s, 25 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestBooleanRewrites

   [junit4] Completed [49/438] on J1 in 0.95s, 7 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.bkd.Test2BBKDPoints

   [junit4] IGNOR/A 0.00s J1 | Test2BBKDPoints.test2D

   [junit4]    > Assumption #1: 'monster' test group is disabled (@Monster(value=takes at least 4 hours and consumes many GB of temp disk space))

   [junit4] IGNOR/A 0.00s J1 | Test2BBKDPoints.test1D

   [junit4]    > Assumption #1: 'monster' test group is disabled (@Monster(value=takes at least 4 hours and consumes many GB of temp disk space))

   [junit4] Completed [50/438] on J1 in 0.03s, 2 tests, 2 skipped

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.MultiCollectorTest

   [junit4] Completed [51/438] on J1 in 0.06s, 4 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestPointValues

   [junit4] Completed [52/438] on J0 in 12.37s, 32 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestTimSorter

   [junit4] Completed [53/438] on J0 in 0.32s, 9 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestSwappedIndexFiles

   [junit4] Completed [54/438] on J0 in 0.20s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestDateSort

   [junit4] Completed [55/438] on J0 in 0.07s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestNotDocIdSet

   [junit4] Completed [56/438] on J0 in 2.38s, 6 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestIntroSorter

   [junit4] Completed [57/438] on J0 in 0.18s, 9 tests

   [junit4] 

   [junit4] HEARTBEAT J2 PID(30170@localhost): 2016-07-30T08:01:37, stalled for 63.2s at: TestIndexWriterThreadsToSegments.testSegmentCountOnFlushRandom

   [junit4] Suite: org.apache.lucene.index.TestIndexWriter

   [junit4] Completed [58/438] on J0 in 13.06s, 71 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestMatchAllDocsQuery

   [junit4] Completed [59/438] on J0 in 0.06s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestTragicIndexWriterDeadlock

   [junit4] Completed [60/438] on J0 in 0.06s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestDocCount

   [junit4] Completed [61/438] on J0 in 0.38s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.similarities.TestSimilarity2

   [junit4] Completed [62/438] on J0 in 0.57s, 8 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestIndexWriterThreadsToSegments

   [junit4] Completed [63/438] on J2 in 78.09s, 4 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestSumDocFreq

   [junit4] Completed [64/438] on J2 in 4.04s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.store.TestSimpleFSLockFactory

   [junit4] Completed [65/438] on J2 in 9.40s, 7 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.store.TestFilterDirectory

   [junit4] IGNOR/A 0.01s J2 | TestFilterDirectory.testFsyncDoesntCreateNewFiles

   [junit4]    > Assumption #1: test only works for FSDirectory subclasses

   [junit4] IGNOR/A 0.01s J2 | TestFilterDirectory.testPendingDeletions

   [junit4]    > Assumption #1: we can only install VirusCheckingFS on an FSDirectory

   [junit4] Completed [66/438] on J2 in 1.48s, 46 tests, 2 skipped

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestConcurrentMergeScheduler

   [junit4] Completed [67/438] on J2 in 17.45s, 16 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.similarities.TestBM25Similarity

   [junit4] Completed [68/438] on J2 in 0.08s, 3 tests

   [junit4] 

   [junit4] HEARTBEAT J1 PID(30169@localhost): 2016-07-30T08:02:25, stalled for 62.6s at: TestControlledRealTimeReopenThread.testControlledRealTimeReopenThread

   [junit4] Suite: org.apache.lucene.util.TestRoaringDocIdSet

   [junit4] Completed [69/438] on J2 in 1.07s, 5 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestDocumentWriter

   [junit4] Completed [70/438] on J2 in 0.19s, 6 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestSloppyPhraseQuery

   [junit4] Completed [71/438] on J2 in 3.14s, 9 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestTerm

   [junit4] Completed [72/438] on J2 in 0.02s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.automaton.TestOperations

   [junit4] Completed [73/438] on J2 in 1.84s, 5 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestRollingUpdates

   [junit4] Completed [74/438] on J2 in 28.98s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestTermsEnum2

   [junit4] Completed [75/438] on J2 in 3.00s, 4 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestNorms

   [junit4] Completed [76/438] on J2 in 6.86s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.codecs.TestCodecLoadingDeadlock

   [junit4] Completed [77/438] on J2 in 1.07s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestDocBoost

   [junit4] Completed [78/438] on J2 in 0.08s, 1 test

   [junit4] 

   [junit4] HEARTBEAT J1 PID(30169@localhost): 2016-07-30T08:03:25, stalled for  123s at: TestControlledRealTimeReopenThread.testControlledRealTimeReopenThread

   [junit4] HEARTBEAT J0 PID(30154@localhost): 2016-07-30T08:03:49, stalled for 60.1s at: TestDemoParallelLeafReader.testRandomMultipleSchemaGensSameField

   [junit4] Suite: org.apache.lucene.util.TestOfflineSorter

   [junit4] Completed [79/438] on J2 in 49.20s, 14 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.spans.TestSpanNearQuery

   [junit4] Completed [80/438] on J2 in 0.04s, 4 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestMaxTermFrequency

   [junit4] Completed [81/438] on J2 in 1.10s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestIndexFileDeleter

   [junit4] Completed [82/438] on J2 in 1.57s, 10 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestAllFilesCheckIndexHeader

   [junit4] Completed [83/438] on J2 in 3.59s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestMergePolicyWrapper

   [junit4] Completed [84/438] on J2 in 0.02s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestUnicodeUtil

   [junit4] Completed [85/438] on J2 in 2.38s, 5 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestParallelReaderEmptyIndex

   [junit4] Completed [86/438] on J2 in 0.06s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestTieredMergePolicy

   [junit4] Completed [87/438] on J2 in 13.39s, 6 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestMergedIterator

   [junit4] Completed [88/438] on J2 in 0.63s, 25 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestIOUtils

   [junit4] Completed [89/438] on J2 in 0.16s, 18 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestExceedMaxTermLength

   [junit4] Completed [90/438] on J2 in 0.05s, 1 test

   [junit4] 

   [junit4] HEARTBEAT J1 PID(30169@localhost): 2016-07-30T08:04:25, stalled for  183s at: TestControlledRealTimeReopenThread.testControlledRealTimeReopenThread

   [junit4] Suite: org.apache.lucene.search.TestBooleanMinShouldMatch

   [junit4] Completed [91/438] on J2 in 5.66s, 17 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestReaderWrapperDVTypeCheck

   [junit4] Completed [92/438] on J2 in 0.04s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestCodecs

   [junit4] Completed [93/438] on J2 in 1.02s, 3 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestSortedNumericSortField

   [junit4] Completed [94/438] on J2 in 0.20s, 9 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestCheckIndex

   [junit4] Completed [95/438] on J2 in 1.02s, 5 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestDocValuesRewriteMethod

   [junit4] Completed [96/438] on J2 in 8.36s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestAutomatonQueryUnicode

   [junit4] Completed [97/438] on J2 in 0.05s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestLazyProxSkipping

   [junit4] Completed [98/438] on J2 in 0.65s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestConsistentFieldNumbers

   [junit4] Completed [99/438] on J2 in 8.09s, 4 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestIndexInput

   [junit4] Completed [100/438] on J2 in 1.57s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestSubScorerFreqs

   [junit4] Completed [101/438] on J2 in 0.10s, 3 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.store.TestMultiMMap

   [junit4] Completed [102/438] on J2 in 15.88s, 56 tests

   [junit4] 

   [junit4] HEARTBEAT J0 PID(30154@localhost): 2016-07-30T08:05:13, stalled for 63.0s at: TestDemoParallelLeafReader.testRandomMultipleSchemaGens

   [junit4] Suite: org.apache.lucene.index.TestDemoParallelLeafReader

   [junit4] Completed [103/438] on J0 in 203.17s, 5 tests

   [junit4] 

   [junit4] HEARTBEAT J1 PID(30169@localhost): 2016-07-30T08:05:25, stalled for  243s at: TestControlledRealTimeReopenThread.testControlledRealTimeReopenThread

   [junit4] Suite: org.apache.lucene.search.TestNumericRangeQuery64

   [junit4] Completed [104/438] on J0 in 23.78s, 29 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestFilterWeight

   [junit4] Completed [105/438] on J0 in 0.01s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestNoDeletionPolicy

   [junit4] Completed [106/438] on J0 in 0.18s, 4 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestDocValuesIndexing

   [junit4] Completed [107/438] on J0 in 0.62s, 34 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestNumericRangeQuery32

   [junit4] Completed [108/438] on J2 in 52.83s, 23 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestStressDeletes

   [junit4] Completed [109/438] on J2 in 2.07s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestBlendedTermQuery

   [junit4] Completed [110/438] on J2 in 0.08s, 3 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.spans.TestSpanBoostQuery

   [junit4] Completed [111/438] on J2 in 0.05s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.store.TestNativeFSLockFactory

   [junit4] Completed [112/438] on J2 in 9.87s, 10 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestNot

   [junit4] Completed [113/438] on J2 in 0.06s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestIndexWriterMaxDocs

   [junit4] IGNOR/A 0.00s J2 | TestIndexWriterMaxDocs.testExactlyAtTrueLimit

   [junit4]    > Assumption #1: 'monster' test group is disabled (@Monster(value=takes over two hours))

   [junit4] Completed [114/438] on J2 in 12.49s, 21 tests, 1 skipped

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestIndexableField

   [junit4] Completed [115/438] on J2 in 0.99s, 2 tests

   [junit4] 

   [junit4] HEARTBEAT J1 PID(30169@localhost): 2016-07-30T08:06:25, stalled for  303s at: TestControlledRealTimeReopenThread.testControlledRealTimeReopenThread

   [junit4] Suite: org.apache.lucene.search.TestControlledRealTimeReopenThread

   [junit4] IGNOR/A 0.00s J1 | TestControlledRealTimeReopenThread.testCRTReopen

   [junit4]    > Assumption #1: 'awaitsfix' test group is disabled (@AwaitsFix(bugUrl=https://issues.apache.org/jira/browse/LUCENE-5737))

   [junit4] Completed [116/438] on J1 in 316.73s, 6 tests, 1 skipped

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestStressAdvance

   [junit4] Completed [117/438] on J1 in 2.98s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestIndexingSequenceNumbers

   [junit4] Completed [118/438] on J0 in 111.64s, 8 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.spans.TestNearSpansOrdered

   [junit4] Completed [119/438] on J0 in 0.29s, 19 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.Test2BPagedBytes

   [junit4] IGNOR/A 0.00s J0 | Test2BPagedBytes.test

   [junit4]    > Assumption #1: 'monster' test group is disabled (@Monster(value=You must increase heap to > 2 G to run this))

   [junit4] Completed [120/438] on J0 in 0.00s, 1 test, 1 skipped

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestSimpleExplanationsWithFillerDocs

   [junit4] IGNOR/A 0.00s J0 | TestSimpleExplanationsWithFillerDocs.testMA2

   [junit4]    > Assumption #1: got: <[test is not viable with empty filler docs, null]>, expected: each not null

   [junit4] IGNOR/A 0.00s J0 | TestSimpleExplanationsWithFillerDocs.testMA1

   [junit4]    > Assumption #1: got: <[test is not viable with empty filler docs, null]>, expected: each not null

   [junit4] Completed [121/438] on J0 in 5.78s, 70 tests, 2 skipped

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestBooleanCoord

   [junit4] Completed [122/438] on J0 in 0.16s, 76 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestLiveFieldValues

   [junit4] Completed [123/438] on J0 in 1.52s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.spans.TestSpansEnum

   [junit4] Completed [124/438] on J0 in 0.06s, 7 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestBytesRef

   [junit4] Completed [125/438] on J0 in 0.01s, 3 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestSimpleExplanations

   [junit4] Completed [126/438] on J0 in 0.99s, 70 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestManyFields

   [junit4] Completed [127/438] on J0 in 0.91s, 3 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestNeverDelete

   [junit4] Completed [128/438] on J0 in 5.24s, 1 test

   [junit4] 

   [junit4] HEARTBEAT J1 PID(30169@localhost): 2016-07-30T08:07:49, stalled for 61.0s at: TestGrowableByteArrayDataOutput.testWriteLargeStrings

   [junit4] Suite: org.apache.lucene.codecs.compressing.TestGrowableByteArrayDataOutput

   [junit4] Completed [129/438] on J1 in 80.16s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestRegexpQuery

   [junit4] Completed [130/438] on J1 in 0.17s, 7 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestSloppyMath

   [junit4] Completed [131/438] on J1 in 1.78s, 8 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.store.TestSimpleFSDirectory

   [junit4] Completed [132/438] on J1 in 0.88s, 44 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestReadOnlyIndex

   [junit4] Completed [133/438] on J1 in 0.28s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestIndexWriterLockRelease

   [junit4] Completed [134/438] on J1 in 0.04s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.Test2BNumericDocValues

   [junit4] IGNOR/A 0.00s J1 | Test2BNumericDocValues.testNumerics

   [junit4]    > Assumption #1: 'monster' test group is disabled (@Monster(value=takes ~ 2 hours if the heap is 5gb))

   [junit4] Completed [135/438] on J1 in 0.01s, 1 test, 1 skipped

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestSameTokenSamePosition

   [junit4] Completed [136/438] on J1 in 0.03s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.automaton.TestAutomaton

   [junit4] Completed [137/438] on J1 in 13.00s, 56 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestThreadedForceMerge

   [junit4] Completed [138/438] on J1 in 0.79s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestDocValuesScoring

   [junit4] Completed [139/438] on J1 in 0.03s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestSetOnce

   [junit4] Completed [140/438] on J1 in 0.02s, 4 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestPagedBytes

   [junit4] Completed [141/438] on J1 in 2.44s, 4 tests, 1 skipped

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestIndexCommit

   [junit4] Completed [142/438] on J1 in 0.01s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestMultiset

   [junit4] Completed [143/438] on J1 in 0.02s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestTwoPhaseCommitTool

   [junit4] Completed [144/438] on J1 in 0.01s, 3 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestDoc

   [junit4] Completed [145/438] on J1 in 0.12s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.spans.TestFilterSpans

   [junit4] Completed [146/438] on J1 in 0.02s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestBinaryTerms

   [junit4] Completed [147/438] on J1 in 0.26s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestRegexpRandom

   [junit4] Completed [148/438] on J1 in 0.16s, 1 test

   [junit4] 

   [junit4] HEARTBEAT J2 PID(30170@localhost): 2016-07-30T08:08:25, stalled for 63.2s at: TestLucene53NormsFormat.testNCommonBig

   [junit4] Suite: org.apache.lucene.search.TestMinShouldMatch2

   [junit4] Completed [149/438] on J1 in 4.42s, 6 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestParallelTermEnum

   [junit4] Completed [150/438] on J1 in 0.03s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestDocumentsWriterDeleteQueue

   [junit4] Completed [151/438] on J1 in 0.11s, 5 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestIndexWriterOutOfFileDescriptors

   [junit4] Completed [152/438] on J1 in 7.44s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.spans.TestSpanContainQuery

   [junit4] Completed [153/438] on J1 in 0.16s, 4 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestCharsRefBuilder

   [junit4] Completed [154/438] on J1 in 0.02s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestSortRescorer

   [junit4] Completed [155/438] on J1 in 0.95s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.codecs.lucene53.TestLucene53NormsFormat

   [junit4] IGNOR/A 0.00s J2 | TestLucene53NormsFormat.testMergeStability

   [junit4]    > Assumption #1: The MockRandom PF randomizes content on the fly, so we can't check it

   [junit4] Completed [156/438] on J2 in 153.48s, 17 tests, 1 skipped

   [junit4] 

   [junit4] Suite: org.apache.lucene.analysis.TestWordlistLoader

   [junit4] Completed [157/438] on J2 in 0.02s, 3 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestMultiThreadTermVectors

   [junit4] Completed [158/438] on J2 in 3.02s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestMultiValuedNumericRangeQuery

   [junit4] Completed [159/438] on J2 in 2.02s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestBoolean2

   [junit4] Completed [160/438] on J1 in 39.77s, 11 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestIndexWriterMerging

   [junit4] Completed [161/438] on J2 in 14.04s, 6 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestMultiTermConstantScore

   [junit4] Completed [162/438] on J1 in 3.40s, 8 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.store.TestLockFactory

   [junit4] Completed [163/438] on J1 in 0.06s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestFieldValueQuery

   [junit4] Completed [164/438] on J2 in 6.04s, 6 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.spans.TestSpanFirstQuery

   [junit4] Completed [165/438] on J2 in 0.02s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestOrdinalMap

   [junit4] Completed [166/438] on J2 in 1.05s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestStressIndexing2

   [junit4] Completed [167/438] on J2 in 3.20s, 3 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.codecs.lucene62.TestLucene62SegmentInfoFormat

   [junit4] Completed [168/438] on J2 in 0.48s, 16 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestSimilarityProvider

   [junit4] Completed [169/438] on J2 in 0.01s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestFixedBitSet

   [junit4] Completed [170/438] on J1 in 22.30s, 19 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestSortRandom

   [junit4] Completed [171/438] on J1 in 1.63s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestMaxPosition

   [junit4] Completed [172/438] on J1 in 0.07s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestParallelLeafReader

   [junit4] Completed [173/438] on J1 in 0.51s, 9 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.codecs.lucene50.TestLucene50StoredFieldsFormat

   [junit4] Completed [174/438] on J2 in 19.57s, 17 tests

   [junit4] 

   [junit4] HEARTBEAT J0 PID(30154@localhost): 2016-07-30T08:09:50, stalled for 69.2s at: TestLucene54DocValuesFormat.testSortedSetVariableLengthBigVsStoredFields

   [junit4] Suite: org.apache.lucene.codecs.lucene60.TestLucene60PointsFormat

   [junit4] IGNOR/A 0.01s J1 | TestLucene60PointsFormat.testMergeStability

   [junit4]    > Assumption #1: merge is not stable

   [junit4] Completed [175/438] on J1 in 36.33s, 17 tests, 1 skipped

   [junit4] 

   [junit4] Suite: org.apache.lucene.codecs.compressing.TestFastCompressionMode

   [junit4] Completed [176/438] on J1 in 1.07s, 11 tests

   [junit4] 

   [junit4] HEARTBEAT J2 PID(30170@localhost): 2016-07-30T08:10:49, stalled for 62.6s at: TestBagOfPostings.test

   [junit4] Suite: org.apache.lucene.index.TestBagOfPostings

   [junit4] Completed [177/438] on J2 in 65.67s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestMultiCollector

   [junit4] Completed [178/438] on J2 in 0.67s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.analysis.TestCharacterUtils

   [junit4] Completed [179/438] on J2 in 0.01s, 4 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestReqExclBulkScorer

   [junit4] Completed [180/438] on J2 in 0.03s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.document.TestField

   [junit4] Completed [181/438] on J2 in 0.04s, 27 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestPayloadsOnVectors

   [junit4] Completed [182/438] on J2 in 0.04s, 3 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestFilterLeafReader

   [junit4] Completed [183/438] on J2 in 0.02s, 3 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.fst.TestBytesStore

   [junit4] Completed [184/438] on J2 in 8.60s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestSearchWithThreads

   [junit4] Completed [185/438] on J1 in 53.54s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestSimpleSearchEquivalence

   [junit4] Completed [186/438] on J1 in 7.57s, 17 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.packed.TestPackedInts

   [junit4] Completed [187/438] on J2 in 29.98s, 28 tests, 2 skipped

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestAddIndexes

   [junit4] Completed [188/438] on J2 in 6.85s, 25 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestMixedDocValuesUpdates

   [junit4] Completed [189/438] on J1 in 31.32s, 4 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestSameScoresWithThreads

   [junit4] Completed [190/438] on J1 in 1.16s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.store.TestNIOFSDirectory

   [junit4] Completed [191/438] on J1 in 2.38s, 44 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestTermdocPerf

   [junit4] Completed [192/438] on J1 in 0.03s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestBooleanQuery

   [junit4] Completed [193/438] on J1 in 0.74s, 22 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestShardSearching

   [junit4] Completed [194/438] on J2 in 19.66s, 1 test

   [junit4] 

   [junit4] HEARTBEAT J0 PID(30154@localhost): 2016-07-30T08:13:02, stalled for 69.3s at: TestLucene54DocValuesFormat.testSparseDocValuesVsStoredFields

   [junit4] HEARTBEAT J2 PID(30170@localhost): 2016-07-30T08:13:25, stalled for 63.5s at: TestIndexWriterExceptions.testTooManyTokens

   [junit4] Suite: org.apache.lucene.codecs.perfield.TestPerFieldDocValuesFormat

   [junit4] Completed [195/438] on J1 in 97.41s, 96 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestUsageTrackingFilterCachingPolicy

   [junit4] Completed [196/438] on J1 in 0.01s, 2 tests

   [junit4] 

   [junit4] HEARTBEAT J0 PID(30154@localhost): 2016-07-30T08:14:02, stalled for  129s at: TestLucene54DocValuesFormat.testSparseDocValuesVsStoredFields

   [junit4] Suite: org.apache.lucene.codecs.lucene50.TestLucene50TermVectorsFormat

   [junit4] Completed [197/438] on J1 in 48.12s, 17 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestAllFilesHaveChecksumFooter

   [junit4] Completed [198/438] on J1 in 0.17s, 1 test

   [junit4] 

   [junit4] HEARTBEAT J2 PID(30170@localhost): 2016-07-30T08:14:25, stalled for  124s at: TestIndexWriterExceptions.testTooManyTokens

   [junit4] Suite: org.apache.lucene.search.TestScorerPerf

   [junit4] Completed [199/438] on J1 in 7.68s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.analysis.TestStopFilter

   [junit4] Completed [200/438] on J1 in 0.02s, 4 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestTopDocsCollector

   [junit4] Completed [201/438] on J1 in 0.15s, 8 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.codecs.perfield.TestPerFieldPostingsFormat

   [junit4] IGNOR/A 0.00s J1 | TestPerFieldPostingsFormat.testPostingsEnumReuse

   [junit4]    > Assumption #1: The MockRandom PF randomizes content on the fly, so we can't check it

   [junit4] IGNOR/A 0.00s J1 | TestPerFieldPostingsFormat.testMergeStability

   [junit4]    > Assumption #1: The MockRandom PF randomizes content on the fly, so we can't check it

   [junit4] Completed [202/438] on J1 in 25.50s, 24 tests, 2 skipped

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestTryDelete

   [junit4] Completed [203/438] on J1 in 0.04s, 3 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestIndexWriterCommit

   [junit4] Completed [204/438] on J1 in 2.36s, 14 tests

   [junit4] 

   [junit4] HEARTBEAT J0 PID(30154@localhost): 2016-07-30T08:15:02, stalled for  189s at: TestLucene54DocValuesFormat.testSparseDocValuesVsStoredFields

   [junit4] Suite: org.apache.lucene.index.TestDeletionPolicy

   [junit4] Completed [205/438] on J1 in 5.08s, 6 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestIndexWriterExceptions

   [junit4] Completed [206/438] on J2 in 205.28s, 36 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestFlushByRamOrCountsPolicy

   [junit4] Completed [207/438] on J2 in 1.87s, 5 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestIndexWriterDelete

   [junit4] Completed [208/438] on J2 in 26.83s, 27 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestIndexWriterMergePolicy

   [junit4] Completed [209/438] on J2 in 3.93s, 7 tests

   [junit4] 

   [junit4] HEARTBEAT J0 PID(30154@localhost): 2016-07-30T08:16:02, stalled for  249s at: TestLucene54DocValuesFormat.testSparseDocValuesVsStoredFields

   [junit4] Suite: org.apache.lucene.search.TestBooleanOr

   [junit4] Completed [210/438] on J1 in 63.17s, 7 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestDuelingCodecs

   [junit4] Completed [211/438] on J1 in 19.55s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestDirectoryReaderReopen

   [junit4] Completed [212/438] on J1 in 2.40s, 14 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestTimeLimitingCollector

   [junit4] Completed [213/438] on J1 in 6.37s, 7 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.codecs.compressing.TestHighCompressionMode

   [junit4] Completed [214/438] on J1 in 1.99s, 7 tests

   [junit4] 

   [junit4] HEARTBEAT J2 PID(30170@localhost): 2016-07-30T08:17:02, stalled for 63.7s at: TestNRTThreads.testNRTThreads

   [junit4] HEARTBEAT J0 PID(30154@localhost): 2016-07-30T08:17:02, stalled for  309s at: TestLucene54DocValuesFormat.testSparseDocValuesVsStoredFields

   [junit4] HEARTBEAT J1 PID(30169@localhost): 2016-07-30T08:17:50, stalled for 71.4s at: TestDocumentsWriterStallControl.testAccquireReleaseRace

   [junit4] HEARTBEAT J2 PID(30170@localhost): 2016-07-30T08:18:02, stalled for  124s at: TestNRTThreads.testNRTThreads

   [junit4] HEARTBEAT J1 PID(30169@localhost): 2016-07-30T08:18:50, stalled for  131s at: TestDocumentsWriterStallControl.testAccquireReleaseRace

   [junit4] HEARTBEAT J2 PID(30170@localhost): 2016-07-30T08:19:02, stalled for  184s at: TestNRTThreads.testNRTThreads

   [junit4] Suite: org.apache.lucene.codecs.lucene54.TestLucene54DocValuesFormat

   [junit4] Completed [215/438] on J0 in 692.42s, 109 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.analysis.tokenattributes.TestCharTermAttributeImpl

   [junit4] Completed [216/438] on J0 in 0.45s, 12 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestSearchAfter

   [junit4] Completed [217/438] on J0 in 1.84s, 1 test

   [junit4] 

   [junit4] HEARTBEAT J1 PID(30169@localhost): 2016-07-30T08:19:50, stalled for  192s at: TestDocumentsWriterStallControl.testAccquireReleaseRace

   [junit4] HEARTBEAT J2 PID(30170@localhost): 2016-07-30T08:20:02, stalled for  244s at: TestNRTThreads.testNRTThreads

   [junit4] HEARTBEAT J0 PID(30154@localhost): 2016-07-30T08:20:26, stalled for 66.2s at: TestSearcherManager.testSearcherManager

   [junit4] HEARTBEAT J1 PID(30169@localhost): 2016-07-30T08:20:50, stalled for  252s at: TestDocumentsWriterStallControl.testAccquireReleaseRace

   [junit4] HEARTBEAT J2 PID(30170@localhost): 2016-07-30T08:21:02, stalled for  304s at: TestNRTThreads.testNRTThreads

   [junit4] HEARTBEAT J0 PID(30154@localhost): 2016-07-30T08:21:26, stalled for  126s at: TestSearcherManager.testSearcherManager

   [junit4] Suite: org.apache.lucene.index.TestNRTThreads

   [junit4] Completed [218/438] on J2 in 332.43s, 1 test

   [junit4] 

   [junit4] HEARTBEAT J1 PID(30169@localhost): 2016-07-30T08:21:50, stalled for  312s at: TestDocumentsWriterStallControl.testAccquireReleaseRace

   [junit4] Suite: org.apache.lucene.search.TestTopDocsMerge

   [junit4] Completed [219/438] on J2 in 24.87s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestLongPostings

   [junit4] Completed [220/438] on J2 in 4.03s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestStressIndexing

   [junit4] Completed [221/438] on J2 in 5.00s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestSnapshotDeletionPolicy

   [junit4] Completed [222/438] on J2 in 1.43s, 7 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestIndexWriterForceMerge

   [junit4] Completed [223/438] on J2 in 3.91s, 4 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestArrayUtil

   [junit4] Completed [224/438] on J2 in 5.88s, 14 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.automaton.TestLevenshteinAutomata

   [junit4] Completed [225/438] on J2 in 1.77s, 4 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestByteSlices

   [junit4] Completed [226/438] on J2 in 7.28s, 1 test

   [junit4] 

   [junit4] HEARTBEAT J0 PID(30154@localhost): 2016-07-30T08:22:26, stalled for  186s at: TestSearcherManager.testSearcherManager

   [junit4] Suite: org.apache.lucene.store.TestBufferedIndexInput

   [junit4] Completed [227/438] on J2 in 0.78s, 4 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestDocumentsWriterStallControl

   [junit4] Completed [228/438] on J1 in 365.81s, 3 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestDirectoryReader

   [junit4] Completed [229/438] on J1 in 3.15s, 24 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestSegmentMerger

   [junit4] Completed [230/438] on J1 in 0.80s, 3 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestPhraseQuery

   [junit4] Completed [231/438] on J1 in 1.66s, 21 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestWeakIdentityMap

   [junit4] Completed [232/438] on J2 in 23.16s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestCollectionUtil

   [junit4] Completed [233/438] on J1 in 16.40s, 4 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestOmitNorms

   [junit4] Completed [234/438] on J1 in 2.41s, 5 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestRegexpRandom2

   [junit4] Completed [235/438] on J1 in 5.48s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.store.TestNRTCachingDirectory

   [junit4] IGNOR/A 0.00s J1 | TestNRTCachingDirectory.testFsyncDoesntCreateNewFiles

   [junit4]    > Assumption #1: test only works for FSDirectory subclasses

   [junit4] IGNOR/A 0.00s J1 | TestNRTCachingDirectory.testPendingDeletions

   [junit4]    > Assumption #1: we can only install VirusCheckingFS on an FSDirectory

   [junit4] Completed [236/438] on J1 in 2.70s, 46 tests, 2 skipped

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestSegmentReader

   [junit4] Completed [237/438] on J1 in 1.08s, 7 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.codecs.compressing.TestFastDecompressionMode

   [junit4] Completed [238/438] on J1 in 3.35s, 11 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestParallelCompositeReader

   [junit4] Completed [239/438] on J1 in 0.84s, 15 tests

   [junit4] 

   [junit4] HEARTBEAT J0 PID(30154@localhost): 2016-07-30T08:23:26, stalled for  246s at: TestSearcherManager.testSearcherManager

   [junit4] Suite: org.apache.lucene.index.TestIndexWriterNRTIsCurrent

   [junit4] Completed [240/438] on J1 in 3.25s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestTransactions

   [junit4] Completed [241/438] on J1 in 2.52s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.codecs.perfield.TestPerFieldPostingsFormat2

   [junit4] Completed [242/438] on J1 in 5.02s, 5 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestSort

   [junit4] Completed [243/438] on J1 in 0.41s, 25 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestBagOfPositions

   [junit4] Completed [244/438] on J2 in 46.67s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.analysis.TestToken

   [junit4] Completed [245/438] on J2 in 0.11s, 5 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestCustomSearcherSort

   [junit4] Completed [246/438] on J2 in 8.31s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestMultiDocValues

   [junit4] Completed [247/438] on J1 in 11.33s, 8 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.similarities.TestSimilarityBase

   [junit4] Completed [248/438] on J2 in 1.58s, 32 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestLegacyNumericUtils

   [junit4] Completed [249/438] on J1 in 2.16s, 12 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.automaton.TestDeterminizeLexicon

   [junit4] Completed [250/438] on J2 in 3.49s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.spans.TestSpans

   [junit4] Completed [251/438] on J2 in 0.81s, 26 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestPostingsOffsets

   [junit4] Completed [252/438] on J2 in 3.05s, 13 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestMixedCodecs

   [junit4] Completed [253/438] on J2 in 4.04s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestTransactionRollback

   [junit4] Completed [254/438] on J2 in 0.48s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestFlex

   [junit4] Completed [255/438] on J2 in 0.60s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.spans.TestSpanExplanations

   [junit4] Completed [256/438] on J2 in 0.60s, 30 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.store.TestRAMDirectory

   [junit4] IGNOR/A 0.00s J2 | TestRAMDirectory.testPendingDeletions

   [junit4]    > Assumption #1: we can only install VirusCheckingFS on an FSDirectory

   [junit4] IGNOR/A 0.01s J2 | TestRAMDirectory.testFsyncDoesntCreateNewFiles

   [junit4]    > Assumption #1: test only works for FSDirectory subclasses

   [junit4] Completed [257/438] on J2 in 1.37s, 48 tests, 2 skipped

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestMultiPhraseQuery

   [junit4] Completed [258/438] on J2 in 0.41s, 17 tests, 1 skipped

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.automaton.TestDeterminism

   [junit4] Completed [259/438] on J2 in 1.94s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestDocsAndPositions

   [junit4] Completed [260/438] on J2 in 1.04s, 6 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.TestMergeSchedulerExternal

   [junit4] Completed [261/438] on J2 in 0.58s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestTermVectorsWriter

   [junit4] Completed [262/438] on J2 in 0.33s, 15 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestOmitPositions

   [junit4] Completed [263/438] on J2 in 0.79s, 4 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestBytesRefHash

   [junit4] Completed [264/438] on J2 in 3.42s, 8 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestIndexWriterOnDiskFull

   [junit4] Completed [265/438] on J1 in 35.36s, 4 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestDisjunctionMaxQuery

   [junit4] Completed [266/438] on J1 in 0.25s, 11 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestForceMergeForever

   [junit4] Completed [267/438] on J1 in 0.97s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestOmitTf

   [junit4] Completed [268/438] on J1 in 0.49s, 6 tests

   [junit4] 

   [junit4] HEARTBEAT J0 PID(30154@localhost): 2016-07-30T08:24:26, stalled for  306s at: TestSearcherManager.testSearcherManager

   [junit4] Suite: org.apache.lucene.index.TestMultiFields

   [junit4] Completed [269/438] on J1 in 1.49s, 3 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestTermRangeQuery

   [junit4] Completed [270/438] on J1 in 0.21s, 7 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestCrashCausesCorruptIndex

   [junit4] Completed [271/438] on J1 in 0.04s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestForTooMuchCloning

   [junit4] Completed [272/438] on J1 in 0.08s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestFuzzyQuery

   [junit4] Completed [273/438] on J1 in 0.09s, 10 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestPayloads

   [junit4] Completed [274/438] on J1 in 0.35s, 7 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestWildcard

   [junit4] Completed [275/438] on J1 in 0.07s, 8 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.spans.TestFieldMaskingSpanQuery

   [junit4] IGNOR/A 0.00s J1 | TestFieldMaskingSpanQuery.testSimple2

   [junit4]    > Assumption #1: Broken scoring: LUCENE-3723

   [junit4] IGNOR/A 0.00s J1 | TestFieldMaskingSpanQuery.testSpans2

   [junit4]    > Assumption #1: Broken scoring: LUCENE-3723

   [junit4] Completed [276/438] on J1 in 0.22s, 11 tests, 2 skipped

   [junit4] 

   [junit4] Suite: org.apache.lucene.TestExternalCodecs

   [junit4] Completed [277/438] on J1 in 0.83s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestSimpleExplanationsOfNonMatches

   [junit4] Completed [278/438] on J1 in 0.05s, 70 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestTermVectorsReader

   [junit4] Completed [279/438] on J1 in 0.13s, 13 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestPerSegmentDeletes

   [junit4] Completed [280/438] on J1 in 0.02s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestPrefixInBooleanQuery

   [junit4] Completed [281/438] on J1 in 0.36s, 4 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestSegmentTermEnum

   [junit4] Completed [282/438] on J1 in 0.11s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestUniqueTermCount

   [junit4] Completed [283/438] on J1 in 0.04s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestComplexExplanationsOfNonMatches

   [junit4] Completed [284/438] on J1 in 0.14s, 21 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestPrefixCodedTerms

   [junit4] Completed [285/438] on J1 in 0.35s, 5 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestAllFilesHaveCodecHeader

   [junit4] Completed [286/438] on J1 in 0.46s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestAutomatonQuery

   [junit4] Completed [287/438] on J1 in 1.47s, 8 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.document.TestDocument

   [junit4] Completed [288/438] on J1 in 0.16s, 12 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestTopFieldCollector

   [junit4] Completed [289/438] on J1 in 0.13s, 7 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestWildcardRandom

   [junit4] Completed [290/438] on J1 in 0.38s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestPrefixRandom

   [junit4] Completed [291/438] on J1 in 0.79s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestMultiLevelSkipList

   [junit4] Completed [292/438] on J1 in 0.08s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestSloppyPhraseQuery2

   [junit4] Completed [293/438] on J2 in 23.61s, 9 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.spans.TestSpanMultiTermQueryWrapper

   [junit4] Completed [294/438] on J1 in 0.07s, 9 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.TestSearchForDuplicates

   [junit4] Completed [295/438] on J2 in 0.15s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestSizeBoundedForceMerge

   [junit4] Completed [296/438] on J1 in 0.11s, 11 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestConstantScoreQuery

   [junit4] Completed [297/438] on J1 in 0.07s, 5 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.automaton.TestCompiledAutomaton

   [junit4] Completed [298/438] on J2 in 0.33s, 6 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestElevationComparator

   [junit4] Completed [299/438] on J1 in 0.07s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestIndexWriterConfig

   [junit4] Completed [300/438] on J2 in 0.03s, 8 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestConjunctions

   [junit4] Completed [301/438] on J1 in 0.01s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestMultiTermQueryRewrites

   [junit4] Completed [302/438] on J2 in 0.03s, 3 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestPositionIncrement

   [junit4] Completed [303/438] on J1 in 0.06s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestSimilarity

   [junit4] Completed [304/438] on J2 in 0.01s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.spans.TestSpanExplanationsOfNonMatches

   [junit4] Completed [305/438] on J1 in 0.04s, 30 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestVirtualMethod

   [junit4] Completed [306/438] on J2 in 0.01s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestRollingBuffer

   [junit4] Completed [307/438] on J1 in 0.10s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestSmallFloat

   [junit4] Completed [308/438] on J2 in 0.12s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.analysis.TestNumericTokenStream

   [junit4] Completed [309/438] on J1 in 0.03s, 6 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.TestSearch

   [junit4] Completed [310/438] on J2 in 0.01s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.analysis.TestCachingTokenFilter

   [junit4] Completed [311/438] on J1 in 0.01s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestFilterIterator

   [junit4] Completed [312/438] on J2 in 0.02s, 8 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.document.TestBinaryDocument

   [junit4] Completed [313/438] on J2 in 0.01s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestAttributeSource

   [junit4] Completed [314/438] on J2 in 0.01s, 7 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestRecyclingByteBlockAllocator

   [junit4] Completed [315/438] on J2 in 0.03s, 3 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestRamUsageEstimator

   [junit4] IGNOR/A 0.00s J2 | TestRamUsageEstimator.testPrintValues

   [junit4]    > Assumption #1: Specify -Dtests.verbose=true to print constants of RamUsageEstimator.

   [junit4] Completed [316/438] on J2 in 0.01s, 5 tests, 1 skipped

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestIsCurrent

   [junit4] Completed [317/438] on J2 in 0.02s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestNoMergeScheduler

   [junit4] Completed [318/438] on J2 in 0.01s, 3 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestNamedSPILoader

   [junit4] Completed [319/438] on J2 in 0.01s, 3 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestPhrasePrefixQuery

   [junit4] Completed [320/438] on J2 in 0.01s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestByteBlockPool

   [junit4] Completed [321/438] on J2 in 0.10s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.TestDemo

   [junit4] Completed [322/438] on J2 in 0.02s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.TestAssertions

   [junit4] Completed [323/438] on J2 in 0.01s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.analysis.TestCharFilter

   [junit4] Completed [324/438] on J2 in 0.01s, 4 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestRollback

   [junit4] Completed [325/438] on J2 in 0.01s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestCloseableThreadLocal

   [junit4] Completed [326/438] on J2 in 0.01s, 3 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestIndexWriterOnJRECrash

   [junit4] Completed [327/438] on J2 in 4.85s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestSearcherManager

   [junit4] Completed [328/438] on J0 in 340.78s, 10 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestMergeRateLimiter

   [junit4] Completed [329/438] on J0 in 0.03s, 1 test

   [junit4] 

   [junit4] HEARTBEAT J2 PID(30170@localhost): 2016-07-30T08:25:50, stalled for 61.4s at: Test4GBStoredFields.test

   [junit4] HEARTBEAT J1 PID(30169@localhost): 2016-07-30T08:25:50, stalled for 70.1s at: Test2BPostings.test

   [junit4] HEARTBEAT J0 PID(30154@localhost): 2016-07-30T08:26:02, stalled for 61.6s at: TestPointQueries.testRandomBinaryBig

   [junit4] HEARTBEAT J1 PID(30169@localhost): 2016-07-30T08:26:50, stalled for  130s at: Test2BPostings.test

   [junit4] HEARTBEAT J2 PID(30170@localhost): 2016-07-30T08:26:50, stalled for  121s at: Test4GBStoredFields.test

   [junit4] HEARTBEAT J1 PID(30169@localhost): 2016-07-30T08:27:50, stalled for  190s at: Test2BPostings.test

   [junit4] HEARTBEAT J2 PID(30170@localhost): 2016-07-30T08:27:50, stalled for  181s at: Test4GBStoredFields.test

   [junit4] Suite: org.apache.lucene.search.TestPointQueries

   [junit4] Completed [330/438] on J0 in 189.68s, 46 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.store.TestDirectory

   [junit4] Completed [331/438] on J0 in 0.40s, 3 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestStressNRT

   [junit4] Completed [332/438] on J0 in 0.22s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestBooleanScorer

   [junit4] Completed [333/438] on J0 in 12.06s, 5 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.document.TestDateTools

   [junit4] Completed [334/438] on J0 in 0.01s, 5 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestTermScorer

   [junit4] Completed [335/438] on J0 in 0.06s, 4 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.codecs.lucene50.TestLucene50StoredFieldsFormatHighCompression

   [junit4] Completed [336/438] on J0 in 26.13s, 19 tests

   [junit4] 

   [junit4] HEARTBEAT J2 PID(30170@localhost): 2016-07-30T08:28:50, stalled for  241s at: Test4GBStoredFields.test

   [junit4] HEARTBEAT J1 PID(30169@localhost): 2016-07-30T08:28:50, stalled for  250s at: Test2BPostings.test

   [junit4] Suite: org.apache.lucene.index.TestCustomNorms

   [junit4] Completed [337/438] on J0 in 2.41s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestPrefixQuery

   [junit4] Completed [338/438] on J0 in 14.30s, 3 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.analysis.standard.TestStandardAnalyzer

   [junit4] Completed [339/438] on J0 in 6.88s, 34 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestTermQuery

   [junit4] Completed [340/438] on J0 in 0.07s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.document.TestFieldType

   [junit4] Completed [341/438] on J0 in 0.02s, 3 tests

   [junit4] 

   [junit4] HEARTBEAT J2 PID(30170@localhost): 2016-07-30T08:29:50, stalled for  301s at: Test4GBStoredFields.test

   [junit4] HEARTBEAT J1 PID(30169@localhost): 2016-07-30T08:29:50, stalled for  310s at: Test2BPostings.test

   [junit4] HEARTBEAT J0 PID(30154@localhost): 2016-07-30T08:30:26, stalled for 62.6s at: TestTermsEnum.testVaryingTermsPerSegment

   [junit4] HEARTBEAT J2 PID(30170@localhost): 2016-07-30T08:30:50, stalled for  361s at: Test4GBStoredFields.test

   [junit4] HEARTBEAT J1 PID(30169@localhost): 2016-07-30T08:30:50, stalled for  370s at: Test2BPostings.test

   [junit4] HEARTBEAT J0 PID(30154@localhost): 2016-07-30T08:31:26, stalled for  123s at: TestTermsEnum.testVaryingTermsPerSegment

   [junit4] Suite: org.apache.lucene.index.Test2BPostings

   [junit4] Completed [342/438] on J1 in 423.65s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestCodecHoldsOpenFiles

   [junit4] Completed [343/438] on J1 in 0.11s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestReaderClosed

   [junit4] Completed [344/438] on J1 in 0.03s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestScoreCachingWrappingScorer

   [junit4] Completed [345/438] on J1 in 0.01s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestIntsRef

   [junit4] Completed [346/438] on J1 in 0.01s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.Test2BPositions

   [junit4] IGNOR/A 0.00s J1 | Test2BPositions.test

   [junit4]    > Assumption #1: 'monster' test group is disabled (@Monster(value=uses lots of space and takes a few minutes))

   [junit4] Completed [347/438] on J1 in 0.00s, 1 test, 1 skipped

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.Test2BPostingsBytes

   [junit4] IGNOR/A 0.00s J1 | Test2BPostingsBytes.test

   [junit4]    > Assumption #1: 'monster' test group is disabled (@Monster(value=takes ~20GB-30GB of space and 10 minutes))

   [junit4] Completed [348/438] on J1 in 0.00s, 1 test, 1 skipped

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestPositiveScoresOnlyCollector

   [junit4] Completed [349/438] on J1 in 0.06s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.Test2BTerms

   [junit4] IGNOR/A 0.00s J1 | Test2BTerms.test2BTerms

   [junit4]    > Assumption #1: 'monster' test group is disabled (@Monster(value=very slow, use 5g minimum heap))

   [junit4] Completed [350/438] on J1 in 0.00s, 1 test, 1 skipped

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestNewestSegment

   [junit4] Completed [351/438] on J1 in 0.01s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.store.TestByteArrayDataInput

   [junit4] Completed [352/438] on J1 in 0.01s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.Test2BBinaryDocValues

   [junit4] IGNOR/A 0.00s J1 | Test2BBinaryDocValues.testVariableBinary

   [junit4]    > Assumption #1: 'monster' test group is disabled (@Monster(value=takes ~ 6 hours if the heap is 5gb))

   [junit4] IGNOR/A 0.00s J1 | Test2BBinaryDocValues.testFixedBinary

   [junit4]    > Assumption #1: 'monster' test group is disabled (@Monster(value=takes ~ 6 hours if the heap is 5gb))

   [junit4] Completed [353/438] on J1 in 0.00s, 2 tests, 2 skipped

   [junit4] 

   [junit4] Suite: org.apache.lucene.analysis.TestCharArrayMap

   [junit4] Completed [354/438] on J1 in 0.09s, 4 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.analysis.TestCharArraySet

   [junit4] Completed [355/438] on J1 in 0.03s, 15 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.analysis.TestReusableStringReader

   [junit4] Completed [356/438] on J1 in 0.01s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.analysis.tokenattributes.TestBytesRefAttImpl

   [junit4] Completed [357/438] on J1 in 0.01s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.analysis.tokenattributes.TestPackedTokenAttributeImpl

   [junit4] Completed [358/438] on J1 in 0.03s, 4 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.codecs.TestCodecUtil

   [junit4] Completed [359/438] on J1 in 0.03s, 15 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.Test4GBStoredFields

   [junit4] Completed [360/438] on J2 in 420.67s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.codecs.lucene50.TestBlockPostingsFormat2

   [junit4] Completed [361/438] on J2 in 1.17s, 4 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.codecs.lucene50.TestBlockPostingsFormat3

   [junit4] Completed [362/438] on J2 in 4.99s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.codecs.lucene50.TestForUtil

   [junit4] Completed [363/438] on J2 in 0.07s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.codecs.lucene50.TestLucene50CompoundFormat

   [junit4] Completed [364/438] on J2 in 3.63s, 26 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.codecs.lucene50.TestLucene50FieldInfoFormat

   [junit4] Completed [365/438] on J2 in 0.34s, 11 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.geo.TestGeoEncodingUtils

   [junit4] Completed [366/438] on J2 in 0.78s, 3 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.codecs.lucene50.TestBlockPostingsFormat

   [junit4] Completed [367/438] on J1 in 29.73s, 26 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.geo.TestPolygon

   [junit4] Completed [368/438] on J1 in 0.03s, 15 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.geo.TestGeoUtils

   [junit4] Completed [369/438] on J2 in 19.20s, 6 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.Test2BPoints

   [junit4] IGNOR/A 0.00s J2 | Test2BPoints.test2D

   [junit4]    > Assumption #1: 'monster' test group is disabled (@Monster(value=takes at least 4 hours and consumes many GB of temp disk space))

   [junit4] IGNOR/A 0.00s J2 | Test2BPoints.test1D

   [junit4]    > Assumption #1: 'monster' test group is disabled (@Monster(value=takes at least 4 hours and consumes many GB of temp disk space))

   [junit4] Completed [370/438] on J2 in 0.01s, 2 tests, 2 skipped

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.Test2BSortedDocValuesFixedSorted

   [junit4] IGNOR/A 0.00s J2 | Test2BSortedDocValuesFixedSorted.testFixedSorted

   [junit4]    > Assumption #1: 'monster' test group is disabled (@Monster(value=Takes ~ 6 hours if the heap is 5gb))

   [junit4] Completed [371/438] on J2 in 0.00s, 1 test, 1 skipped

   [junit4] 

   [junit4] HEARTBEAT J0 PID(30154@localhost): 2016-07-30T08:32:26, stalled for  183s at: TestTermsEnum.testVaryingTermsPerSegment

   [junit4] Suite: org.apache.lucene.index.TestTermsEnum

   [junit4] Completed [372/438] on J0 in 195.21s, 11 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestIndexWriterExceptions2

   [junit4] Completed [373/438] on J0 in 10.36s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestDocInverterPerFieldErrorInfo

   [junit4] Completed [374/438] on J0 in 0.07s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.geo.TestPolygon2D

   [junit4] Completed [375/438] on J1 in 26.58s, 10 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestExitableDirectoryReader

   [junit4] Completed [376/438] on J1 in 0.67s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestFilterDirectoryReader

   [junit4] Completed [377/438] on J1 in 0.05s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestIndexReaderClose

   [junit4] Completed [378/438] on J1 in 1.02s, 2 tests

   [junit4] 

   [junit4] HEARTBEAT J2 PID(30170@localhost): 2016-07-30T08:33:38, stalled for 69.4s at: TestBinaryDocValuesUpdates.testTonsOfUpdates

   [junit4] HEARTBEAT J0 PID(30154@localhost): 2016-07-30T08:33:50, stalled for 71.4s at: TestDuelingCodecsAtNight.testBigEquals

   [junit4] HEARTBEAT J1 PID(30169@localhost): 2016-07-30T08:34:14, stalled for 62.8s at: TestIndexSorting.testRandom3

   [junit4] HEARTBEAT J2 PID(30170@localhost): 2016-07-30T08:34:38, stalled for  129s at: TestBinaryDocValuesUpdates.testTonsOfUpdates

   [junit4] HEARTBEAT J0 PID(30154@localhost): 2016-07-30T08:34:50, stalled for  131s at: TestDuelingCodecsAtNight.testBigEquals

   [junit4] HEARTBEAT J1 PID(30169@localhost): 2016-07-30T08:35:14, stalled for  123s at: TestIndexSorting.testRandom3

   [junit4] Suite: org.apache.lucene.index.TestBinaryDocValuesUpdates

   [junit4] Completed [379/438] on J2 in 178.44s, 28 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestIndexWriterFromReader

   [junit4] Completed [380/438] on J2 in 0.95s, 12 tests

   [junit4] 

   [junit4] HEARTBEAT J0 PID(30154@localhost): 2016-07-30T08:35:50, stalled for  192s at: TestDuelingCodecsAtNight.testBigEquals

   [junit4] HEARTBEAT J1 PID(30169@localhost): 2016-07-30T08:36:14, stalled for  183s at: TestIndexSorting.testRandom3

   [junit4] HEARTBEAT J2 PID(30170@localhost): 2016-07-30T08:36:26, stalled for 61.5s at: TestIndexWriterOnVMError.testCheckpoint

   [junit4] HEARTBEAT J0 PID(30154@localhost): 2016-07-30T08:36:50, stalled for  252s at: TestDuelingCodecsAtNight.testBigEquals

   [junit4] HEARTBEAT J1 PID(30169@localhost): 2016-07-30T08:37:14, stalled for  243s at: TestIndexSorting.testRandom3

   [junit4] Suite: org.apache.lucene.index.TestDuelingCodecsAtNight

   [junit4] Completed [381/438] on J0 in 286.33s, 3 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestInfoStream

   [junit4] Completed [382/438] on J0 in 0.01s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestLogMergePolicy

   [junit4] Completed [383/438] on J0 in 0.02s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestMultiTermsEnum

   [junit4] Completed [384/438] on J0 in 0.02s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestSegmentInfos

   [junit4] Completed [385/438] on J0 in 0.01s, 3 tests

   [junit4] 

   [junit4

[...truncated too long message...]

sts-6.x/solr/core/test-lib/apacheds-interceptors-schema-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-subtree-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-trigger-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-jdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-jdbm1-2.0.0-M2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-kerberos-codec-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-ldif-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-mavibot-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-protocol-kerberos-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-protocol-ldap-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-protocol-shared-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-xdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/api-all-1.0.0-M20.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/bcprov-jdk15-1.45.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/cglib-nodep-2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/commons-collections-3.2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/commons-math3-3.4.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/easymock-3.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/ehcache-core-2.4.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/hadoop-common-2.7.2-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/hadoop-hdfs-2.7.2-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/hadoop-minikdc-2.7.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/htrace-core-3.2.0-incubating.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jackson-annotations-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jackson-core-asl-1.9.13.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jackson-databind-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jackson-mapper-asl-1.9.13.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jersey-core-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jersey-server-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jetty-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jetty-util-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/mina-core-2.0.0-M5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/mockito-core-1.9.5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/netty-all-4.0.36.Final.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/objenesis-1.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/presto-parser-0.122.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/slice-0.10.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/analysis/icu/lucene-analyzers-icu-6.2.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/contrib/solr-analysis-extras/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/queryparser/classes/test:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/contrib/analysis-extras/lib/icu4j-56.1.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-launcher.jar:/x1/jenkins/.ant/lib/ivy-2.3.0.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-antlr.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-regexp.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-resolver.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-net.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jmf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-oro.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-xalan2.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-logging.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jsch.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit4.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-javamail.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jai.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jdepend.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bsf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-swing.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-netrexx.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bcel.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-testutil.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-log4j.jar:/x1/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.8/lib/tools.jar:/x1/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.3.4.jar com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/temp/junit4-J0-20160730_103644_416.events @/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/temp/junit4-J0-20160730_103644_416.suites -stdin
   [junit4] ERROR: JVM J0 ended with an exception: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1501)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:121)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:955)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:952)
   [junit4] 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4] 	at java.lang.Thread.run(Thread.java:745)

BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:771: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:707: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/common-build.xml:536: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/common-build.xml:1443: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/common-build.xml:984: At least one slave process threw an exception, first: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.

Total time: 264 minutes 0 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
ERROR: Failed to archive {heapdumps/java_pid1074.hprof=heapdumps/java_pid1074.hprof, README.txt=README.txt, solr/build/solr-core/test/temp/junit4-J1-20160730_103644_416.events=solr/build/solr-core/test/temp/junit4-J1-20160730_103644_416.events, solr/build/solr-core/test/temp/junit4-J2-20160730_103644_416.events=solr/build/solr-core/test/temp/junit4-J2-20160730_103644_416.events, solr/build/solr-core/test/temp/junit4-J0-20160730_103644_416.events=solr/build/solr-core/test/temp/junit4-J0-20160730_103644_416.events} due to internal error; falling back to full archiving
java.lang.NullPointerException
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any





[JENKINS] Lucene-Solr-NightlyTests-6.x - Build # 130 - Failure

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-6.x/130/

No tests ran.

Build Log:
[...truncated 27 lines...]
ERROR: Step ‘Publish JUnit test result report’ failed: No test report files were found. Configuration error?
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any




[JENKINS] Lucene-Solr-NightlyTests-6.x - Build # 129 - Still Unstable

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-6.x/129/

5 tests failed.
FAILED:  org.apache.solr.cloud.ForceLeaderTest.testReplicasInLIRNoLeader

Error Message:
No live SolrServers available to handle this request:[http://127.0.0.1:50211/_/hb/forceleader_test_collection_shard1_replica3]

Stack Trace:
org.apache.solr.client.solrj.impl.CloudSolrClient$RouteException: No live SolrServers available to handle this request:[http://127.0.0.1:50211/_/hb/forceleader_test_collection_shard1_replica3]
	at __randomizedtesting.SeedInfo.seed([EA7083120FCEF20D:CE7B7D2364C0B6C]:0)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.directUpdate(CloudSolrClient.java:760)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1172)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1061)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:997)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.sendDocsWithRetry(AbstractFullDistribZkTestBase.java:753)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.sendDocsWithRetry(AbstractFullDistribZkTestBase.java:741)
	at org.apache.solr.cloud.ForceLeaderTest.sendDoc(ForceLeaderTest.java:424)
	at org.apache.solr.cloud.ForceLeaderTest.testReplicasInLIRNoLeader(ForceLeaderTest.java:131)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:50211/_/hb/forceleader_test_collection_shard1_replica3]
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:383)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.lambda$directUpdate$0(CloudSolrClient.java:735)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	... 1 more
Caused by: org.apache.solr.client.solrj.SolrServerException: Server refused connection at: http://127.0.0.1:50211/_/hb/forceleader_test_collection_shard1_replica3
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:597)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:259)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:248)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:403)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:372)
	... 6 more
Caused by: java.net.ConnectException: Connection refused
	at java.net.PlainSocketImpl.socketConnect(Native Method)
	at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
	at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
	at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
	at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
	at java.net.Socket.connect(Socket.java:589)
	at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:117)
	at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:177)
	at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:304)
	at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:611)
	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:446)
	at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:495)
	... 10 more


FAILED:  org.apache.solr.core.OpenCloseCoreStressTest.test10Minutes

Error Message:
Captured an uncaught exception in thread: Thread[id=11598, name=Lucene Merge Thread #5, state=RUNNABLE, group=TGRP-OpenCloseCoreStressTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=11598, name=Lucene Merge Thread #5, state=RUNNABLE, group=TGRP-OpenCloseCoreStressTest]
Caused by: org.apache.lucene.index.MergePolicy$MergeException: java.lang.IllegalStateException: this writer hit an unrecoverable error; cannot merge
	at __randomizedtesting.SeedInfo.seed([EA7083120FCEF20D]:0)
	at org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:668)
	at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:648)
Caused by: java.lang.IllegalStateException: this writer hit an unrecoverable error; cannot merge
	at org.apache.lucene.index.IndexWriter._mergeInit(IndexWriter.java:4046)
	at org.apache.lucene.index.IndexWriter.mergeInit(IndexWriter.java:4026)
	at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3880)
	at org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:588)
	at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:626)
Caused by: java.nio.file.FileSystemException: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_EA7083120FCEF20D-001/index-SimpleFSDirectory-004/_4b_TestBloomFilteredLucenePostings_0.tip: Too many open files
	at org.apache.lucene.mockfile.HandleLimitFS.onOpen(HandleLimitFS.java:48)
	at org.apache.lucene.mockfile.HandleTrackingFS.callOpenHook(HandleTrackingFS.java:81)
	at org.apache.lucene.mockfile.HandleTrackingFS.newByteChannel(HandleTrackingFS.java:271)
	at java.nio.file.Files.newByteChannel(Files.java:361)
	at java.nio.file.Files.newByteChannel(Files.java:407)
	at org.apache.lucene.store.SimpleFSDirectory.openInput(SimpleFSDirectory.java:77)
	at org.apache.lucene.util.LuceneTestCase.slowFileExists(LuceneTestCase.java:2695)
	at org.apache.lucene.store.MockDirectoryWrapper.openInput(MockDirectoryWrapper.java:749)
	at org.apache.lucene.codecs.blocktree.BlockTreeTermsReader.<init>(BlockTreeTermsReader.java:176)
	at org.apache.lucene.codecs.lucene50.Lucene50PostingsFormat.fieldsProducer(Lucene50PostingsFormat.java:445)
	at org.apache.lucene.codecs.bloom.BloomFilteringPostingsFormat$BloomFilteredFieldsProducer.<init>(BloomFilteringPostingsFormat.java:174)
	at org.apache.lucene.codecs.bloom.BloomFilteringPostingsFormat.fieldsProducer(BloomFilteringPostingsFormat.java:150)
	at org.apache.lucene.codecs.bloom.TestBloomFilteredLucenePostings.fieldsProducer(TestBloomFilteredLucenePostings.java:71)
	at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsReader.<init>(PerFieldPostingsFormat.java:261)
	at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat.fieldsProducer(PerFieldPostingsFormat.java:341)
	at org.apache.lucene.index.SegmentCoreReaders.<init>(SegmentCoreReaders.java:106)
	at org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:74)
	at org.apache.lucene.index.ReadersAndUpdates.getReader(ReadersAndUpdates.java:145)
	at org.apache.lucene.index.BufferedUpdatesStream$SegmentState.<init>(BufferedUpdatesStream.java:384)
	at org.apache.lucene.index.BufferedUpdatesStream.openSegmentStates(BufferedUpdatesStream.java:416)
	at org.apache.lucene.index.BufferedUpdatesStream.applyDeletesAndUpdates(BufferedUpdatesStream.java:261)
	at org.apache.lucene.index.IndexWriter._mergeInit(IndexWriter.java:4068)
	... 4 more


FAILED:  org.apache.solr.core.OpenCloseCoreStressTest.test15Seconds

Error Message:
Captured an uncaught exception in thread: Thread[id=11830, name=Lucene Merge Thread #11, state=RUNNABLE, group=TGRP-OpenCloseCoreStressTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=11830, name=Lucene Merge Thread #11, state=RUNNABLE, group=TGRP-OpenCloseCoreStressTest]
Caused by: org.apache.lucene.index.MergePolicy$MergeException: java.nio.file.FileSystemException: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_EA7083120FCEF20D-001/index-SimpleFSDirectory-014/_ap_LuceneFixedGap_0.tii: Too many open files
	at __randomizedtesting.SeedInfo.seed([EA7083120FCEF20D]:0)
	at org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:668)
	at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:648)
Caused by: java.nio.file.FileSystemException: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_EA7083120FCEF20D-001/index-SimpleFSDirectory-014/_ap_LuceneFixedGap_0.tii: Too many open files
	at org.apache.lucene.mockfile.HandleLimitFS.onOpen(HandleLimitFS.java:48)
	at org.apache.lucene.mockfile.HandleTrackingFS.callOpenHook(HandleTrackingFS.java:81)
	at org.apache.lucene.mockfile.HandleTrackingFS.newByteChannel(HandleTrackingFS.java:271)
	at java.nio.file.Files.newByteChannel(Files.java:361)
	at java.nio.file.Files.newByteChannel(Files.java:407)
	at org.apache.lucene.store.SimpleFSDirectory.openInput(SimpleFSDirectory.java:77)
	at org.apache.lucene.util.LuceneTestCase.slowFileExists(LuceneTestCase.java:2695)
	at org.apache.lucene.store.MockDirectoryWrapper.openInput(MockDirectoryWrapper.java:749)
	at org.apache.lucene.codecs.blockterms.FixedGapTermsIndexReader.<init>(FixedGapTermsIndexReader.java:71)
	at org.apache.lucene.codecs.blockterms.LuceneFixedGap.fieldsProducer(LuceneFixedGap.java:101)
	at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsReader.<init>(PerFieldPostingsFormat.java:261)
	at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat.fieldsProducer(PerFieldPostingsFormat.java:341)
	at org.apache.lucene.index.SegmentCoreReaders.<init>(SegmentCoreReaders.java:106)
	at org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:74)
	at org.apache.lucene.index.ReadersAndUpdates.getReader(ReadersAndUpdates.java:145)
	at org.apache.lucene.index.BufferedUpdatesStream$SegmentState.<init>(BufferedUpdatesStream.java:384)
	at org.apache.lucene.index.BufferedUpdatesStream.openSegmentStates(BufferedUpdatesStream.java:416)
	at org.apache.lucene.index.BufferedUpdatesStream.applyDeletesAndUpdates(BufferedUpdatesStream.java:261)
	at org.apache.lucene.index.IndexWriter._mergeInit(IndexWriter.java:4068)
	at org.apache.lucene.index.IndexWriter.mergeInit(IndexWriter.java:4026)
	at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3880)
	at org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:588)
	at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:626)


FAILED:  junit.framework.TestSuite.org.apache.solr.handler.TestReplicationHandler

Error Message:
ObjectTracker found 3 object(s) that were not released!!! [NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory]

Stack Trace:
java.lang.AssertionError: ObjectTracker found 3 object(s) that were not released!!! [NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory]
	at __randomizedtesting.SeedInfo.seed([EA7083120FCEF20D]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.junit.Assert.assertNull(Assert.java:551)
	at org.apache.solr.SolrTestCaseJ4.teardownTestCases(SolrTestCaseJ4.java:258)
	at sun.reflect.GeneratedMethodAccessor49.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:834)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=7766, name=collection2, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=7766, name=collection2, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:40243: collection already exists: awholynewstresscollection_collection2_1
	at __randomizedtesting.SeedInfo.seed([EA7083120FCEF20D]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:590)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:259)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:248)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:403)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:356)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1291)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1061)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:997)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1599)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1620)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:987)




Build Log:
[...truncated 11378 lines...]
   [junit4] Suite: org.apache.solr.cloud.ForceLeaderTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.cloud.ForceLeaderTest_EA7083120FCEF20D-001/init-core-data-001
   [junit4]   2> 472255 INFO  (SUITE-ForceLeaderTest-seed#[EA7083120FCEF20D]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=https://issues.apache.org/jira/browse/SOLR-5776)
   [junit4]   2> 472255 INFO  (SUITE-ForceLeaderTest-seed#[EA7083120FCEF20D]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /_/hb
   [junit4]   2> 472256 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 472279 INFO  (Thread-2752) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 472279 INFO  (Thread-2752) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 472379 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.ZkTestServer start zk server on port:42388
   [junit4]   2> 472379 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 472398 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 472507 INFO  (zkCallback-700-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@19218242 name:ZooKeeperConnection Watcher:127.0.0.1:42388 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 472507 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 472507 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 472507 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 472605 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 472617 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 472661 INFO  (zkCallback-701-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3d343705 name:ZooKeeperConnection Watcher:127.0.0.1:42388/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 472661 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 472661 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 472661 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 472662 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 472662 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 472663 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 472663 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 472664 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 472665 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 472665 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 472666 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 472666 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 472666 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 472667 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 472667 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 472667 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 472668 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 472668 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 472668 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 472668 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 472669 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 472669 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 472670 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 472670 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 472670 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 472670 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 472671 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 472671 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 472748 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.cloud.ForceLeaderTest_EA7083120FCEF20D-001/control-001/cores/collection1
   [junit4]   2> 472752 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.e.j.s.Server jetty-9.3.8.v20160314
   [junit4]   2> 472753 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@6880e69c{/_/hb,null,AVAILABLE}
   [junit4]   2> 472759 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.e.j.s.ServerConnector Started ServerConnector@1d08414f{HTTP/1.1,[http/1.1]}{127.0.0.1:41555}
   [junit4]   2> 472759 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.e.j.s.Server Started @475042ms
   [junit4]   2> 472759 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.cloud.ForceLeaderTest_EA7083120FCEF20D-001/tempDir-001/control/data, hostContext=/_/hb, hostPort=45193, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.cloud.ForceLeaderTest_EA7083120FCEF20D-001/control-001/cores}
   [junit4]   2> 472763 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@73d16e93
   [junit4]   2> 472763 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.cloud.ForceLeaderTest_EA7083120FCEF20D-001/control-001'
   [junit4]   2> 472763 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 472763 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 472763 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 472764 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 472766 INFO  (zkCallback-702-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7e79c4a name:ZooKeeperConnection Watcher:127.0.0.1:42388/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 472768 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 472768 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 472769 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 472769 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.cloud.ForceLeaderTest_EA7083120FCEF20D-001/control-001/solr.xml
   [junit4]   2> 472773 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.cloud.ForceLeaderTest_EA7083120FCEF20D-001/control-001/cores
   [junit4]   2> 472773 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.CoreContainer New CoreContainer 1885481281
   [junit4]   2> 472774 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.cloud.ForceLeaderTest_EA7083120FCEF20D-001/control-001]
   [junit4]   2> 472774 WARN  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.cloud.ForceLeaderTest_EA7083120FCEF20D-001/control-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.cloud.ForceLeaderTest_EA7083120FCEF20D-001/control-001/lib
   [junit4]   2> 472774 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,connectionsEvictorSleepDelay : 5000,maxConnectionIdleTime : 40000,
   [junit4]   2> 472775 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 472776 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 472776 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 472776 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:42388/solr
   [junit4]   2> 472776 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 472776 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 472776 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 472777 INFO  (zkCallback-705-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5efd3b96 name:ZooKeeperConnection Watcher:127.0.0.1:42388 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 472777 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 472777 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 472778 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [n:127.0.0.1:45193__%2Fhb    ] o.a.s.c.ZkController Added new OnReconnect listener org.apache.solr.cloud.ZkController$$Lambda$59/1050007703@56e0da71
   [junit4]   2> 472778 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [n:127.0.0.1:45193__%2Fhb    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 472779 INFO  (zkCallback-706-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7b7e58f4 name:ZooKeeperConnection Watcher:127.0.0.1:42388/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 472779 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [n:127.0.0.1:45193__%2Fhb    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 472780 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [n:127.0.0.1:45193__%2Fhb    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 472780 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [n:127.0.0.1:45193__%2Fhb    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 472781 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [n:127.0.0.1:45193__%2Fhb    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 472782 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [n:127.0.0.1:45193__%2Fhb    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 472783 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [n:127.0.0.1:45193__%2Fhb    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 472783 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [n:127.0.0.1:45193__%2Fhb    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 472784 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [n:127.0.0.1:45193__%2Fhb    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 472784 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [n:127.0.0.1:45193__%2Fhb    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 472784 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [n:127.0.0.1:45193__%2Fhb    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 472785 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [n:127.0.0.1:45193__%2Fhb    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 472785 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [n:127.0.0.1:45193__%2Fhb    ] o.a.s.c.c.ZkStateReader Loaded empty cluster properties
   [junit4]   2> 472785 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [n:127.0.0.1:45193__%2Fhb    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (0)
   [junit4]   2> 472786 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [n:127.0.0.1:45193__%2Fhb    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 472787 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [n:127.0.0.1:45193__%2Fhb    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 472787 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [n:127.0.0.1:45193__%2Fhb    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 472787 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [n:127.0.0.1:45193__%2Fhb    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/96320006612516868-127.0.0.1:45193__%2Fhb-n_0000000000
   [junit4]   2> 472787 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [n:127.0.0.1:45193__%2Fhb    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:45193__%2Fhb
   [junit4]   2> 472787 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [n:127.0.0.1:45193__%2Fhb    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 472788 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [n:127.0.0.1:45193__%2Fhb    ] o.a.s.c.Overseer Overseer (id=96320006612516868-127.0.0.1:45193__%2Fhb-n_0000000000) starting
   [junit4]   2> 472788 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [n:127.0.0.1:45193__%2Fhb    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 472791 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [n:127.0.0.1:45193__%2Fhb    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 472791 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [n:127.0.0.1:45193__%2Fhb    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:45193__%2Fhb
   [junit4]   2> 472791 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [n:127.0.0.1:45193__%2Fhb    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:45193__%2Fhb
   [junit4]   2> 472798 INFO  (zkCallback-706-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [0])
   [junit4]   2> 472798 INFO  (zkCallback-706-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 472798 INFO  (OverseerStateUpdate-96320006612516868-127.0.0.1:45193__%2Fhb-n_0000000000) [n:127.0.0.1:45193__%2Fhb    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 472798 INFO  (OverseerCollectionConfigSetProcessor-96320006612516868-127.0.0.1:45193__%2Fhb-n_0000000000) [n:127.0.0.1:45193__%2Fhb    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 472799 INFO  (OverseerStateUpdate-96320006612516868-127.0.0.1:45193__%2Fhb-n_0000000000) [n:127.0.0.1:45193__%2Fhb    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (1)
   [junit4]   2> 472805 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [n:127.0.0.1:45193__%2Fhb    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 472805 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [n:127.0.0.1:45193__%2Fhb    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 472805 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [n:127.0.0.1:45193__%2Fhb    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.cloud.ForceLeaderTest_EA7083120FCEF20D-001/control-001/cores
   [junit4]   2> 472806 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [n:127.0.0.1:45193__%2Fhb    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, loadOnStartup=true, configSetProperties=configsetprops.json, collection=control_collection, coreNodeName=, dataDir=data/, shard=}
   [junit4]   2> 472806 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [n:127.0.0.1:45193__%2Fhb    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.cloud.ForceLeaderTest_EA7083120FCEF20D-001/control-001/cores/collection1
   [junit4]   2> 472806 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [n:127.0.0.1:45193__%2Fhb    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 472808 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 472808 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 472809 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [n:127.0.0.1:45193__%2Fhb    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0
   [junit4]   2> 472809 INFO  (OverseerStateUpdate-96320006612516868-127.0.0.1:45193__%2Fhb-n_0000000000) [n:127.0.0.1:45193__%2Fhb    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:45193/_/hb",
   [junit4]   2>   "node_name":"127.0.0.1:45193__%2Fhb",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 472809 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [n:127.0.0.1:45193__%2Fhb    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 472809 INFO  (OverseerStateUpdate-96320006612516868-127.0.0.1:45193__%2Fhb-n_0000000000) [n:127.0.0.1:45193__%2Fhb    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:45193/_/hb",
   [junit4]   2>   "node_name":"127.0.0.1:45193__%2Fhb",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 472809 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection   x:collection1] o.a.s.c.ZkController Collection control_collection not visible yet, but flagging it so a watch is registered when it becomes visible
   [junit4]   2> 472809 INFO  (OverseerStateUpdate-96320006612516868-127.0.0.1:45193__%2Fhb-n_0000000000) [n:127.0.0.1:45193__%2Fhb    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 472809 INFO  (OverseerStateUpdate-96320006612516868-127.0.0.1:45193__%2Fhb-n_0000000000) [n:127.0.0.1:45193__%2Fhb    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 472810 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Deleting data for [control_collection]
   [junit4]   2> 472810 INFO  (zkCallback-706-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 472814 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 472814 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 472814 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 472814 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 472814 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/control_collection]
   [junit4]   2> 472814 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/control_collection] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 472814 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.cloud.ForceLeaderTest_EA7083120FCEF20D-001/control-001/cores/collection1'
   [junit4]   2> 472814 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 472815 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 472815 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 472825 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 472827 WARN  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection   x:collection1] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
   [junit4]   2> 472828 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.2.0
   [junit4]   2> 472835 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection   x:collection1] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 472839 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 472983 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 472991 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 472993 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 473002 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 473005 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 473008 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 473009 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 473010 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0
   [junit4]   2> 473010 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 473011 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 473011 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.cloud.ForceLeaderTest_EA7083120FCEF20D-001/control-001/cores/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.cloud.ForceLeaderTest_EA7083120FCEF20D-001/control-001/cores/collection1/data/]
   [junit4]   2> 473011 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@2011ad98
   [junit4]   2> 473011 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.cloud.ForceLeaderTest_EA7083120FCEF20D-001/control-001/cores/collection1/data
   [junit4]   2> 473011 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.cloud.ForceLeaderTest_EA7083120FCEF20D-001/control-001/cores/collection1/data/index/
   [junit4]   2> 473011 WARN  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.cloud.ForceLeaderTest_EA7083120FCEF20D-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 473011 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.cloud.ForceLeaderTest_EA7083120FCEF20D-001/control-001/cores/collection1/data/index
   [junit4]   2> 473012 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=18, maxMergeAtOnceExplicit=50, maxMergedSegmentMB=88.07421875, floorSegmentMB=0.9521484375, forceMergeDeletesPctAllowed=29.137806083283635, segmentsPerTier=27.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 473012 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(NRTCachingDirectory(RAMDirectory@67a5708c lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@be99c73; maxCacheMB=0.43359375 maxMergeSizeMB=0.8662109375)),segFN=segments_1,generation=1}
   [junit4]   2> 473012 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 473013 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.cloud.ForceLeaderTest_EA7083120FCEF20D-001/control-001/cores/collection1/data/
   [junit4]   2> 473013 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.cloud.ForceLeaderTest_EA7083120FCEF20D-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 473015 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 473015 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 473015 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 473015 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 473015 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 473015 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 473015 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 473015 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 473015 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 473015 WARN  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
   [junit4]   2> 473016 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 473017 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 473018 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 473018 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 473020 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/mbeans,standard,/update/json/docs,/admin/luke,/export,/get,/admin/properties,/update/json,/admin/threads,/dump,/update/csv,/sql,/graph,/admin/segments,/admin/system,/replication,/config,/stream,/schema,/admin/plugins,/admin/logging,/admin/ping,/update,/admin/file,/terms
   [junit4]   2> 473020 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 473020 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 473020 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 473021 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 473021 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 473021 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@5c3e6152
   [junit4]   2> 473021 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(NRTCachingDirectory(RAMDirectory@67a5708c lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@be99c73; maxCacheMB=0.43359375 maxMergeSizeMB=0.8662109375)),segFN=segments_1,generation=1}
   [junit4]   2> 473021 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 473022 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@64e8b701[collection1] main]
   [junit4]   2> 473022 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/control_collection]
   [junit4]   2> 473022 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/control_collection] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 473022 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 473022 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 473023 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 473023 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 473023 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 473023 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 473023 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 473023 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 473023 INFO  (searcherExecutor-2012-thread-1-processing-n:127.0.0.1:45193__%2Fhb x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@64e8b701[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 473024 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 473024 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 473024 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 473024 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1541120106569924608
   [junit4]   2> 473025 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 1.0ms to seed version buckets with highest version 1541120106569924608
   [junit4]   2> 473025 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 473026 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 473026 INFO  (coreZkRegister-2005-thread-1-processing-n:127.0.0.1:45193__%2Fhb x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:45193/_/hb collection:control_collection shard:shard1
   [junit4]   2> 473027 INFO  (coreZkRegister-2005-thread-1-processing-n:127.0.0.1:45193__%2Fhb x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 473029 INFO  (coreZkRegister-2005-thread-1-processing-n:127.0.0.1:45193__%2Fhb x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/96320006612516868-core_node1-n_0000000000
   [junit4]   2> 473029 INFO  (coreZkRegister-2005-thread-1-processing-n:127.0.0.1:45193__%2Fhb x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard=shard1 and weAreReplacement=false and leaderVoteWait=10000
   [junit4]   2> 473030 INFO  (coreZkRegister-2005-thread-1-processing-n:127.0.0.1:45193__%2Fhb x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 473031 INFO  (coreZkRegister-2005-thread-1-processing-n:127.0.0.1:45193__%2Fhb x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 473031 INFO  (coreZkRegister-2005-thread-1-processing-n:127.0.0.1:45193__%2Fhb x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:45193/_/hb/collection1/
   [junit4]   2> 473031 INFO  (OverseerStateUpdate-96320006612516868-127.0.0.1:45193__%2Fhb-n_0000000000) [n:127.0.0.1:45193__%2Fhb    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 473031 INFO  (coreZkRegister-2005-thread-1-processing-n:127.0.0.1:45193__%2Fhb x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 473031 INFO  (coreZkRegister-2005-thread-1-processing-n:127.0.0.1:45193__%2Fhb x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:45193/_/hb/collection1/ has no replicas
   [junit4]   2> 473031 INFO  (coreZkRegister-2005-thread-1-processing-n:127.0.0.1:45193__%2Fhb x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 473033 INFO  (coreZkRegister-2005-thread-1-processing-n:127.0.0.1:45193__%2Fhb x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/control_collection/leaders/shard1/leader after winning as /collections/control_collection/leader_elect/shard1/election/96320006612516868-core_node1-n_0000000000
   [junit4]   2> 473034 INFO  (coreZkRegister-2005-thread-1-processing-n:127.0.0.1:45193__%2Fhb x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:45193/_/hb/collection1/ shard1
   [junit4]   2> 473035 INFO  (OverseerStateUpdate-96320006612516868-127.0.0.1:45193__%2Fhb-n_0000000000) [n:127.0.0.1:45193__%2Fhb    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:45193/_/hb",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 1
   [junit4]   2> 473136 INFO  (zkCallback-706-thread-2-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 473185 INFO  (coreZkRegister-2005-thread-1-processing-n:127.0.0.1:45193__%2Fhb x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:45193/_/hb/collection1/ and leader is http://127.0.0.1:45193/_/hb/collection1/
   [junit4]   2> 473185 INFO  (coreZkRegister-2005-thread-1-processing-n:127.0.0.1:45193__%2Fhb x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:45193/_/hb
   [junit4]   2> 473185 INFO  (coreZkRegister-2005-thread-1-processing-n:127.0.0.1:45193__%2Fhb x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 473185 INFO  (coreZkRegister-2005-thread-1-processing-n:127.0.0.1:45193__%2Fhb x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 473185 INFO  (coreZkRegister-2005-thread-1-processing-n:127.0.0.1:45193__%2Fhb x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 473186 INFO  (coreZkRegister-2005-thread-1-processing-n:127.0.0.1:45193__%2Fhb x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Checking legacy cluster state for collection control_collection
   [junit4]   2> 473186 INFO  (OverseerStateUpdate-96320006612516868-127.0.0.1:45193__%2Fhb-n_0000000000) [n:127.0.0.1:45193__%2Fhb    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:45193/_/hb",
   [junit4]   2>   "node_name":"127.0.0.1:45193__%2Fhb",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"} current state version: 2
   [junit4]   2> 473186 INFO  (OverseerStateUpdate-96320006612516868-127.0.0.1:45193__%2Fhb-n_0000000000) [n:127.0.0.1:45193__%2Fhb    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:45193/_/hb",
   [junit4]   2>   "node_name":"127.0.0.1:45193__%2Fhb",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 473289 INFO  (zkCallback-706-thread-2-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 473311 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 473312 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 473313 INFO  (zkCallback-710-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7950a679 name:ZooKeeperConnection Watcher:127.0.0.1:42388/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 473313 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 473313 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 473313 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 473313 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.c.ZkStateReader Loaded empty cluster properties
   [junit4]   2> 473314 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 473314 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 473315 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with stateFormat=2
   [junit4]   2> 473315 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 473322 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 473322 INFO  (zkCallback-711-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3b082b74 name:ZooKeeperConnection Watcher:127.0.0.1:42388/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 473322 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 473322 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 473323 INFO  (OverseerStateUpdate-96320006612516868-127.0.0.1:45193__%2Fhb-n_0000000000) [n:127.0.0.1:45193__%2Fhb    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"create",
   [junit4]   2>   "name":"collection1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "stateFormat":"2"} current state version: 3
   [junit4]   2> 473323 INFO  (OverseerStateUpdate-96320006612516868-127.0.0.1:45193__%2Fhb-n_0000000000) [n:127.0.0.1:45193__%2Fhb    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 473402 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.cloud.ForceLeaderTest_EA7083120FCEF20D-001/shard-1-001/cores/collection1
   [junit4]   2> 473403 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.cloud.ForceLeaderTest_EA7083120FCEF20D-001/shard-1-001
   [junit4]   2> 473403 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.e.j.s.Server jetty-9.3.8.v20160314
   [junit4]   2> 473406 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@484e9764{/_/hb,null,AVAILABLE}
   [junit4]   2> 473406 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.e.j.s.ServerConnector Started ServerConnector@1a0c83a6{HTTP/1.1,[http/1.1]}{127.0.0.1:51260}
   [junit4]   2> 473406 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.e.j.s.Server Started @475689ms
   [junit4]   2> 473406 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.cloud.ForceLeaderTest_EA7083120FCEF20D-001/tempDir-001/jetty1, solrconfig=solrconfig.xml, hostContext=/_/hb, hostPort=59676, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.cloud.ForceLeaderTest_EA7083120FCEF20D-001/shard-1-001/cores}
   [junit4]   2> 473407 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@73d16e93
   [junit4]   2> 473407 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.cloud.ForceLeaderTest_EA7083120FCEF20D-001/shard-1-001'
   [junit4]   2> 473407 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 473407 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 473407 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 473408 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 473408 INFO  (zkCallback-712-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1bb72e0d name:ZooKeeperConnection Watcher:127.0.0.1:42388/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 473408 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 473409 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 473409 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 473409 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.cloud.ForceLeaderTest_EA7083120FCEF20D-001/shard-1-001/solr.xml
   [junit4]   2> 473413 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.cloud.ForceLeaderTest_EA7083120FCEF20D-001/shard-1-001/cores
   [junit4]   2> 473413 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.CoreContainer New CoreContainer 853855013
   [jun

[...truncated too long message...]

ecutor-5173-thread-1) [    x:collection1] o.a.s.u.SolrCoreState Closing SolrCoreState
   [junit4]   2> 1949104 INFO  (coreCloseExecutor-5173-thread-1) [    x:collection1] o.a.s.u.DefaultSolrCoreState SolrCoreState ref count has reached 0 - closing IndexWriter
   [junit4]   2> 1949104 INFO  (coreCloseExecutor-5173-thread-1) [    x:collection1] o.a.s.u.DefaultSolrCoreState closing IndexWriter with IndexWriterCloser
   [junit4]   2> 1949105 INFO  (coreCloseExecutor-5173-thread-1) [    x:collection1] o.a.s.c.SolrCore [collection1] Closing main searcher on request.
   [junit4]   2> 1949105 INFO  (coreCloseExecutor-5173-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing MockDirectoryFactory - 2 directories currently being tracked
   [junit4]   2> 1949105 INFO  (coreCloseExecutor-5173-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory looking to close /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandler_EA7083120FCEF20D-001/solr-instance-030/./collection1/data [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandler_EA7083120FCEF20D-001/solr-instance-030/./collection1/data;done=false>>]
   [junit4]   2> 1949105 INFO  (coreCloseExecutor-5173-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory, CoreContainer#isShutdown=true
   [junit4]   2> 1949105 INFO  (coreCloseExecutor-5173-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandler_EA7083120FCEF20D-001/solr-instance-030/./collection1/data
   [junit4]   2> 1949105 INFO  (coreCloseExecutor-5173-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory looking to close /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandler_EA7083120FCEF20D-001/solr-instance-030/./collection1/data/index [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandler_EA7083120FCEF20D-001/solr-instance-030/./collection1/data/index;done=false>>]
   [junit4]   2> 1949105 INFO  (coreCloseExecutor-5173-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory, CoreContainer#isShutdown=true
   [junit4]   2> 1949105 INFO  (coreCloseExecutor-5173-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandler_EA7083120FCEF20D-001/solr-instance-030/./collection1/data/index
   [junit4]   2> 1949106 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[EA7083120FCEF20D]) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@370cd1b8{/solr,null,UNAVAILABLE}
   [junit4]   2> 1949107 INFO  (SUITE-TestReplicationHandler-seed#[EA7083120FCEF20D]-worker) [    ] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> ObjectTracker found 3 object(s) that were not released!!! [NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory]
   [junit4]   2> org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException
   [junit4]   2> 	at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
   [junit4]   2> 	at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347)
   [junit4]   2> 	at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:367)
   [junit4]   2> 	at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:251)
   [junit4]   2> 	at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:388)
   [junit4]   2> 	at org.apache.solr.handler.ReplicationHandler.lambda$handleRequestBody$0(ReplicationHandler.java:278)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 
   [junit4]   2> org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException
   [junit4]   2> 	at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
   [junit4]   2> 	at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347)
   [junit4]   2> 	at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:367)
   [junit4]   2> 	at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:251)
   [junit4]   2> 	at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:388)
   [junit4]   2> 	at org.apache.solr.handler.ReplicationHandler.lambda$handleRequestBody$0(ReplicationHandler.java:278)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 
   [junit4]   2> org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException
   [junit4]   2> 	at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
   [junit4]   2> 	at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347)
   [junit4]   2> 	at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:367)
   [junit4]   2> 	at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:251)
   [junit4]   2> 	at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:388)
   [junit4]   2> 	at org.apache.solr.handler.ReplicationHandler.lambda$handleRequestBody$0(ReplicationHandler.java:278)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 
   [junit4]   2> NOTE: test params are: codec=FastCompressingStoredFields(storedFieldsFormat=CompressingStoredFieldsFormat(compressionMode=FAST, chunkSize=2, maxDocsPerChunk=6, blockSize=5), termVectorsFormat=CompressingTermVectorsFormat(compressionMode=FAST, chunkSize=2, blockSize=5)), sim=RandomSimilarity(queryNorm=true,coord=no): {}, locale=es-DO, timezone=America/Paramaribo
   [junit4]   2> NOTE: Linux 3.13.0-85-generic amd64/Oracle Corporation 1.8.0_74 (64-bit)/cpus=4,threads=1,free=333089344,total=519045120
   [junit4]   2> NOTE: All tests run in this JVM: [CdcrRequestHandlerTest, RecoveryAfterSoftCommitTest, TestCollationField, CollectionStateFormat2Test, TestSolrQueryParserDefaultOperatorResource, TestIndexSearcher, TestIndexingPerformance, LeaderElectionTest, CoreSorterTest, DirectUpdateHandlerTest, FullSolrCloudDistribCmdsTest, TestOnReconnectListenerSupport, StatelessScriptUpdateProcessorFactoryTest, FullHLLTest, ConjunctionSolrSpellCheckerTest, TestDistributedMissingSort, HdfsChaosMonkeySafeLeaderTest, TestReload, TestGroupingSearch, TestSolrDeletionPolicy1, TestJmxMonitoredMap, InfoHandlerTest, TestSort, TestSuggestSpellingConverter, TestInfoStreamLogging, DistanceUnitsTest, TestNumericTerms64, TestSizeLimitedDistributedMap, BigEndianAscendingWordSerializerTest, TestBlobHandler, TestSchemaSimilarityResource, SampleTest, TestLocalFSCloudBackupRestore, TestFieldSortValues, TestManagedResource, DirectUpdateHandlerOptimizeTest, PKIAuthenticationIntegrationTest, TestDefaultStatsCache, TestPostingsSolrHighlighter, PrimitiveFieldTypeTest, UnloadDistributedZkTest, CoreAdminCreateDiscoverTest, JavabinLoaderTest, OverseerTest, FieldMutatingUpdateProcessorTest, DistributedSuggestComponentTest, TestPHPSerializedResponseWriter, TestCloudSchemaless, TestMiniSolrCloudClusterBase, TestReRankQParserPlugin, TestElisionMultitermQuery, TestHdfsCloudBackupRestore, HdfsUnloadDistributedZkTest, TestReversedWildcardFilterFactory, TestZkChroot, BinaryUpdateRequestHandlerTest, TestExactStatsCache, SliceStateTest, DistributedFacetPivotWhiteBoxTest, TestSolrQueryParser, TestSolrCLIRunExample, TestPerFieldSimilarityClassic, TestFieldCacheWithThreads, HighlighterConfigTest, FieldAnalysisRequestHandlerTest, LukeRequestHandlerTest, TestDefaultSearchFieldResource, TestQueryWrapperFilter, TestSolrJ, TestSearchPerf, TestGraphMLResponseWriter, BadComponentTest, SmileWriterTest, TestFieldCollectionResource, DocValuesMissingTest, TestUseDocValuesAsStored2, TestReplicationHandlerBackup, TestSolrIndexConfig, CircularListTest, SuggestComponentContextFilterQueryTest, RegexBoostProcessorTest, DistributedDebugComponentTest, TermVectorComponentDistributedTest, ClusterStateTest, SimpleFacetsTest, TestClusterStateMutator, TestMissingGroups, TestSolr4Spatial2, SuggesterTest, TestCollapseQParserPlugin, WrapperMergePolicyFactoryTest, TestDistribDocBasedVersion, AutoCommitTest, TestOrdValues, TestTolerantSearch, OverseerRolesTest, ConnectionManagerTest, RecoveryZkTest, TestManagedSchemaAPI, SolrTestCaseJ4Test, UpdateParamsTest, RequestHandlersTest, ChaosMonkeySafeLeaderTest, BasicDistributedZk2Test, ShardRoutingTest, BasicZkTest, TestReplicationHandler]
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestReplicationHandler -Dtests.seed=EA7083120FCEF20D -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=es-DO -Dtests.timezone=America/Paramaribo -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR   0.00s J2 | TestReplicationHandler (suite) <<<
   [junit4]    > Throwable #1: java.lang.AssertionError: ObjectTracker found 3 object(s) that were not released!!! [NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory]
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([EA7083120FCEF20D]:0)
   [junit4]    > 	at org.apache.solr.SolrTestCaseJ4.teardownTestCases(SolrTestCaseJ4.java:258)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4] Completed [312/624 (4!)] on J2 in 584.61s, 15 tests, 1 failure, 1 skipped <<< FAILURES!

[...truncated 61085 lines...]



[JENKINS] Lucene-Solr-NightlyTests-6.x - Build # 128 - Still unstable

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-6.x/128/

2 tests failed.
FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Timeout occured while waiting response from server at: https://127.0.0.1:53671

Stack Trace:
org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: https://127.0.0.1:53671
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:601)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:259)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:248)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.makeRequest(CollectionsAPIDistributedZkTest.java:399)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testErrorHandling(CollectionsAPIDistributedZkTest.java:526)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test(CollectionsAPIDistributedZkTest.java:179)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.net.SocketTimeoutException: Read timed out
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
	at java.net.SocketInputStream.read(SocketInputStream.java:170)
	at java.net.SocketInputStream.read(SocketInputStream.java:141)
	at sun.security.ssl.InputRecord.readFully(InputRecord.java:465)
	at sun.security.ssl.InputRecord.read(InputRecord.java:503)
	at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:973)
	at sun.security.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:930)
	at sun.security.ssl.AppInputStream.read(AppInputStream.java:105)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:160)
	at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:84)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:273)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
	at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
	at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)
	at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)
	at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197)
	at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)
	at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)
	at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685)
	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487)
	at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:495)
	... 47 more


FAILED:  org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.test

Error Message:
Timeout occured while waiting response from server at: http://127.0.0.1:53613

Stack Trace:
org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:53613
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:601)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:259)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:248)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.makeRequest(CollectionsAPIDistributedZkTest.java:399)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testErrorHandling(CollectionsAPIDistributedZkTest.java:526)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test(CollectionsAPIDistributedZkTest.java:179)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.net.SocketTimeoutException: Read timed out
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
	at java.net.SocketInputStream.read(SocketInputStream.java:170)
	at java.net.SocketInputStream.read(SocketInputStream.java:141)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:160)
	at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:84)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:273)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
	at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
	at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)
	at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)
	at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197)
	at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)
	at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)
	at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685)
	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487)
	at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:495)
	... 47 more




Build Log:
[...truncated 11430 lines...]
   [junit4] Suite: org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_1A7B2F875163F270-001/init-core-data-001
   [junit4]   2> 19271 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[1A7B2F875163F270]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
   [junit4]   2> 19272 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[1A7B2F875163F270]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   2> 19810 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[1A7B2F875163F270]-worker) [    ] o.a.h.u.NativeCodeLoader Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
   [junit4]   1> Formatting using clusterid: testClusterID
   [junit4]   2> 20364 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[1A7B2F875163F270]-worker) [    ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
   [junit4]   2> 20499 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[1A7B2F875163F270]-worker) [    ] o.m.log Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
   [junit4]   2> 20518 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[1A7B2F875163F270]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 20569 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[1A7B2F875163F270]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 20617 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[1A7B2F875163F270]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/hdfs to ./temp/Jetty_localhost_59627_hdfs____k0kry9/webapp
   [junit4]   2> 21651 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[1A7B2F875163F270]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:59627
   [junit4]   2> 22477 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[1A7B2F875163F270]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 22485 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[1A7B2F875163F270]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 22494 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[1A7B2F875163F270]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_45570_datanode____.xz2hnw/webapp
   [junit4]   2> 22917 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[1A7B2F875163F270]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:45570
   [junit4]   2> 23235 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[1A7B2F875163F270]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 23236 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[1A7B2F875163F270]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 23248 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[1A7B2F875163F270]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_42028_datanode____bqr28v/webapp
   [junit4]   2> 23959 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[1A7B2F875163F270]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:42028
   [junit4]   2> 24703 INFO  (IPC Server handler 6 on 33325) [    ] BlockStateChange BLOCK* processReport: from storage DS-ba89a504-ec71-4619-a0b0-d71f73c0622a node DatanodeRegistration(127.0.0.1:41506, datanodeUuid=8d84edb7-a412-4611-912d-6827820941a8, infoPort=59269, infoSecurePort=0, ipcPort=38150, storageInfo=lv=-56;cid=testClusterID;nsid=2103673468;c=0), blocks: 0, hasStaleStorage: true, processing time: 2 msecs
   [junit4]   2> 24703 INFO  (IPC Server handler 5 on 33325) [    ] BlockStateChange BLOCK* processReport: from storage DS-1bab66fd-eb12-4458-8cac-c5b99b2231aa node DatanodeRegistration(127.0.0.1:59002, datanodeUuid=f1efbae0-5047-47c2-aa49-4f3aa53855b4, infoPort=47004, infoSecurePort=0, ipcPort=54544, storageInfo=lv=-56;cid=testClusterID;nsid=2103673468;c=0), blocks: 0, hasStaleStorage: true, processing time: 0 msecs
   [junit4]   2> 24705 INFO  (IPC Server handler 6 on 33325) [    ] BlockStateChange BLOCK* processReport: from storage DS-c6bf4594-378f-46e4-82d1-03a82587dc80 node DatanodeRegistration(127.0.0.1:41506, datanodeUuid=8d84edb7-a412-4611-912d-6827820941a8, infoPort=59269, infoSecurePort=0, ipcPort=38150, storageInfo=lv=-56;cid=testClusterID;nsid=2103673468;c=0), blocks: 0, hasStaleStorage: false, processing time: 1 msecs
   [junit4]   2> 24705 INFO  (IPC Server handler 5 on 33325) [    ] BlockStateChange BLOCK* processReport: from storage DS-82c5b856-f875-4878-88de-1da30e9cb5e2 node DatanodeRegistration(127.0.0.1:59002, datanodeUuid=f1efbae0-5047-47c2-aa49-4f3aa53855b4, infoPort=47004, infoSecurePort=0, ipcPort=54544, storageInfo=lv=-56;cid=testClusterID;nsid=2103673468;c=0), blocks: 0, hasStaleStorage: false, processing time: 0 msecs
   [junit4]   2> 24959 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 24959 INFO  (Thread-134) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 24959 INFO  (Thread-134) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 25059 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [    ] o.a.s.c.ZkTestServer start zk server on port:54319
   [junit4]   2> 25059 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 25064 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 25072 INFO  (zkCallback-24-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4db52acc name:ZooKeeperConnection Watcher:127.0.0.1:54319 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 25072 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 25073 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 25073 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 25084 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 25088 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 25110 INFO  (zkCallback-25-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2fe587de name:ZooKeeperConnection Watcher:127.0.0.1:54319/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 25110 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 25110 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 25110 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 25112 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 25113 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 25114 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 25115 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 25115 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 25117 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 25117 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 25124 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 25124 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 25125 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 25125 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 25126 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 25127 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 25128 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 25128 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 25129 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 25129 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 25130 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 25130 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 25131 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 25132 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 25133 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 25133 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 25134 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 25134 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 25140 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 25146 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 25152 INFO  (zkCallback-26-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@26b48ca9 name:ZooKeeperConnection Watcher:127.0.0.1:54319 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 25152 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 25152 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 25152 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 25156 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 25157 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 25164 INFO  (zkCallback-27-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@69cb8039 name:ZooKeeperConnection Watcher:127.0.0.1:54319/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 25164 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 25164 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 25164 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf2/solrconfig.xml
   [junit4]   2> 25164 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/solrconfig.xml
   [junit4]   2> 25168 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf2/schema.xml
   [junit4]   2> 25168 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/schema.xml
   [junit4]   2> 25170 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf2/enumsConfig.xml
   [junit4]   2> 25171 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/enumsConfig.xml
   [junit4]   2> 25172 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf2/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 25172 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 25173 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf2/stopwords.txt
   [junit4]   2> 25173 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/stopwords.txt
   [junit4]   2> 25174 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf2/protwords.txt
   [junit4]   2> 25174 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/protwords.txt
   [junit4]   2> 25175 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf2/currency.xml
   [junit4]   2> 25175 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/currency.xml
   [junit4]   2> 25179 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf2/open-exchange-rates.json
   [junit4]   2> 25179 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/open-exchange-rates.json
   [junit4]   2> 25180 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf2/mapping-ISOLatin1Accent.txt
   [junit4]   2> 25180 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/mapping-ISOLatin1Accent.txt
   [junit4]   2> 25181 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf2/old_synonyms.txt
   [junit4]   2> 25181 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/old_synonyms.txt
   [junit4]   2> 25182 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf2/synonyms.txt
   [junit4]   2> 25183 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/synonyms.txt
   [junit4]   2> 25184 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/elevate.xml to /configs/conf2/elevate.xml
   [junit4]   2> 25184 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/elevate.xml
   [junit4]   2> 25258 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_1A7B2F875163F270-001/control-001/cores/collection1
   [junit4]   2> 25259 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [    ] o.e.j.s.Server jetty-9.3.8.v20160314
   [junit4]   2> 25263 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@c43b82c{/,null,AVAILABLE}
   [junit4]   2> 25264 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [    ] o.e.j.s.ServerConnector Started ServerConnector@1998c1c5{HTTP/1.1,[http/1.1]}{127.0.0.1:36076}
   [junit4]   2> 25264 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [    ] o.e.j.s.Server Started @27373ms
   [junit4]   2> 25264 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=hdfs://localhost:33325/hdfs__localhost_33325__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-6.x_solr_build_solr-core_test_J0_temp_solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_1A7B2F875163F270-001_tempDir-002_control_data, hostContext=/, hostPort=36076, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_1A7B2F875163F270-001/control-001/cores}
   [junit4]   2> 25264 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@73d16e93
   [junit4]   2> 25264 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_1A7B2F875163F270-001/control-001'
   [junit4]   2> 25264 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 25264 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 25265 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 25265 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 25280 INFO  (zkCallback-28-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@20f7efdb name:ZooKeeperConnection Watcher:127.0.0.1:54319/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 25280 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 25280 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 25285 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 25285 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_1A7B2F875163F270-001/control-001/solr.xml
   [junit4]   2> 25290 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_1A7B2F875163F270-001/control-001/cores
   [junit4]   2> 25290 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [    ] o.a.s.c.CoreContainer New CoreContainer 215443763
   [junit4]   2> 25290 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_1A7B2F875163F270-001/control-001]
   [junit4]   2> 25290 WARN  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_1A7B2F875163F270-001/control-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_1A7B2F875163F270-001/control-001/lib
   [junit4]   2> 25291 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,connectionsEvictorSleepDelay : 5000,maxConnectionIdleTime : 40000,
   [junit4]   2> 25292 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 25292 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 25292 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 25292 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:54319/solr
   [junit4]   2> 25292 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 25292 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 25294 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 25298 INFO  (zkCallback-31-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1840c0c name:ZooKeeperConnection Watcher:127.0.0.1:54319 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 25304 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 25304 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 25310 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [n:127.0.0.1:36076_    ] o.a.s.c.ZkController Added new OnReconnect listener org.apache.solr.cloud.ZkController$$Lambda$53/743894380@76d4c765
   [junit4]   2> 25324 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [n:127.0.0.1:36076_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 25338 INFO  (zkCallback-32-thread-1-processing-n:127.0.0.1:36076_) [n:127.0.0.1:36076_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3ba70928 name:ZooKeeperConnection Watcher:127.0.0.1:54319/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 25338 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [n:127.0.0.1:36076_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 25340 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [n:127.0.0.1:36076_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 25342 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [n:127.0.0.1:36076_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 25344 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [n:127.0.0.1:36076_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 25345 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [n:127.0.0.1:36076_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 25346 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [n:127.0.0.1:36076_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 25347 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [n:127.0.0.1:36076_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 25350 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [n:127.0.0.1:36076_    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 25351 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [n:127.0.0.1:36076_    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 25352 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [n:127.0.0.1:36076_    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 25352 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [n:127.0.0.1:36076_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 25352 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [n:127.0.0.1:36076_    ] o.a.s.c.c.ZkStateReader Loaded empty cluster properties
   [junit4]   2> 25353 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [n:127.0.0.1:36076_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (0)
   [junit4]   2> 25355 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [n:127.0.0.1:36076_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 25356 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [n:127.0.0.1:36076_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 25357 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [n:127.0.0.1:36076_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 25357 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [n:127.0.0.1:36076_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/96316726256009222-127.0.0.1:36076_-n_0000000000
   [junit4]   2> 25357 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [n:127.0.0.1:36076_    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:36076_
   [junit4]   2> 25358 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [n:127.0.0.1:36076_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 25358 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [n:127.0.0.1:36076_    ] o.a.s.c.Overseer Overseer (id=96316726256009222-127.0.0.1:36076_-n_0000000000) starting
   [junit4]   2> 25360 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [n:127.0.0.1:36076_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 25364 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [n:127.0.0.1:36076_    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 25365 INFO  (OverseerStateUpdate-96316726256009222-127.0.0.1:36076_-n_0000000000) [n:127.0.0.1:36076_    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 25365 INFO  (OverseerCollectionConfigSetProcessor-96316726256009222-127.0.0.1:36076_-n_0000000000) [n:127.0.0.1:36076_    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 25377 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [n:127.0.0.1:36076_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:36076_
   [junit4]   2> 25377 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [n:127.0.0.1:36076_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:36076_
   [junit4]   2> 25378 INFO  (zkCallback-32-thread-1-processing-n:127.0.0.1:36076_) [n:127.0.0.1:36076_    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [0])
   [junit4]   2> 25386 INFO  (OverseerStateUpdate-96316726256009222-127.0.0.1:36076_-n_0000000000) [n:127.0.0.1:36076_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 25393 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [n:127.0.0.1:36076_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 25393 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [n:127.0.0.1:36076_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 25393 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [n:127.0.0.1:36076_    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_1A7B2F875163F270-001/control-001/cores
   [junit4]   2> 25394 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [n:127.0.0.1:36076_    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, loadOnStartup=true, configSetProperties=configsetprops.json, collection=control_collection, coreNodeName=, dataDir=data/, shard=}
   [junit4]   2> 25394 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [n:127.0.0.1:36076_    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_1A7B2F875163F270-001/control-001/cores/collection1
   [junit4]   2> 25394 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [n:127.0.0.1:36076_    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 25405 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [n:127.0.0.1:36076_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0
   [junit4]   2> 25405 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [n:127.0.0.1:36076_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 25406 INFO  (coreLoadExecutor-71-thread-1-processing-n:127.0.0.1:36076_) [n:127.0.0.1:36076_ c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 25406 INFO  (coreLoadExecutor-71-thread-1-processing-n:127.0.0.1:36076_) [n:127.0.0.1:36076_ c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 25408 INFO  (coreLoadExecutor-71-thread-1-processing-n:127.0.0.1:36076_) [n:127.0.0.1:36076_ c:control_collection   x:collection1] o.a.s.c.ZkController Collection control_collection not visible yet, but flagging it so a watch is registered when it becomes visible
   [junit4]   2> 25408 INFO  (OverseerStateUpdate-96316726256009222-127.0.0.1:36076_-n_0000000000) [n:127.0.0.1:36076_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:36076",
   [junit4]   2>   "node_name":"127.0.0.1:36076_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 25408 INFO  (OverseerStateUpdate-96316726256009222-127.0.0.1:36076_-n_0000000000) [n:127.0.0.1:36076_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:36076",
   [junit4]   2>   "node_name":"127.0.0.1:36076_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 25409 INFO  (OverseerStateUpdate-96316726256009222-127.0.0.1:36076_-n_0000000000) [n:127.0.0.1:36076_    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 25409 INFO  (coreLoadExecutor-71-thread-1-processing-n:127.0.0.1:36076_) [n:127.0.0.1:36076_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Deleting data for [control_collection]
   [junit4]   2> 25409 INFO  (OverseerStateUpdate-96316726256009222-127.0.0.1:36076_-n_0000000000) [n:127.0.0.1:36076_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 25411 INFO  (zkCallback-32-thread-1-processing-n:127.0.0.1:36076_) [n:127.0.0.1:36076_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 25412 INFO  (coreLoadExecutor-71-thread-1-processing-n:127.0.0.1:36076_) [n:127.0.0.1:36076_ c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 26412 INFO  (coreLoadExecutor-71-thread-1-processing-n:127.0.0.1:36076_) [n:127.0.0.1:36076_ c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 26413 INFO  (coreLoadExecutor-71-thread-1-processing-n:127.0.0.1:36076_) [n:127.0.0.1:36076_ c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 26414 INFO  (coreLoadExecutor-71-thread-1-processing-n:127.0.0.1:36076_) [n:127.0.0.1:36076_ c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 26414 INFO  (coreLoadExecutor-71-thread-1-processing-n:127.0.0.1:36076_) [n:127.0.0.1:36076_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/control_collection]
   [junit4]   2> 26415 INFO  (coreLoadExecutor-71-thread-1-processing-n:127.0.0.1:36076_) [n:127.0.0.1:36076_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/control_collection] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 26415 INFO  (coreLoadExecutor-71-thread-1-processing-n:127.0.0.1:36076_) [n:127.0.0.1:36076_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_1A7B2F875163F270-001/control-001/cores/collection1'
   [junit4]   2> 26415 INFO  (coreLoadExecutor-71-thread-1-processing-n:127.0.0.1:36076_) [n:127.0.0.1:36076_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 26415 INFO  (coreLoadExecutor-71-thread-1-processing-n:127.0.0.1:36076_) [n:127.0.0.1:36076_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 26417 INFO  (coreLoadExecutor-71-thread-1-processing-n:127.0.0.1:36076_) [n:127.0.0.1:36076_ c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 26429 INFO  (coreLoadExecutor-71-thread-1-processing-n:127.0.0.1:36076_) [n:127.0.0.1:36076_ c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 26434 WARN  (coreLoadExecutor-71-thread-1-processing-n:127.0.0.1:36076_) [n:127.0.0.1:36076_ c:control_collection   x:collection1] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
   [junit4]   2> 26435 INFO  (coreLoadExecutor-71-thread-1-processing-n:127.0.0.1:36076_) [n:127.0.0.1:36076_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.2.0
   [junit4]   2> 26442 INFO  (coreLoadExecutor-71-thread-1-processing-n:127.0.0.1:36076_) [n:127.0.0.1:36076_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 26450 INFO  (coreLoadExecutor-71-thread-1-processing-n:127.0.0.1:36076_) [n:127.0.0.1:36076_ c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 26543 INFO  (coreLoadExecutor-71-thread-1-processing-n:127.0.0.1:36076_) [n:127.0.0.1:36076_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 26561 INFO  (coreLoadExecutor-71-thread-1-processing-n:127.0.0.1:36076_) [n:127.0.0.1:36076_ c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 26562 INFO  (coreLoadExecutor-71-thread-1-processing-n:127.0.0.1:36076_) [n:127.0.0.1:36076_ c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 26613 INFO  (coreLoadExecutor-71-thread-1-processing-n:127.0.0.1:36076_) [n:127.0.0.1:36076_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 26615 INFO  (coreLoadExecutor-71-thread-1-processing-n:127.0.0.1:36076_) [n:127.0.0.1:36076_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 26616 INFO  (coreLoadExecutor-71-thread-1-processing-n:127.0.0.1:36076_) [n:127.0.0.1:36076_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 26617 INFO  (coreLoadExecutor-71-thread-1-processing-n:127.0.0.1:36076_) [n:127.0.0.1:36076_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 26618 INFO  (coreLoadExecutor-71-thread-1-processing-n:127.0.0.1:36076_) [n:127.0.0.1:36076_ c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0
   [junit4]   2> 26618 INFO  (coreLoadExecutor-71-thread-1-processing-n:127.0.0.1:36076_) [n:127.0.0.1:36076_ c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 26618 INFO  (coreLoadExecutor-71-thread-1-processing-n:127.0.0.1:36076_) [n:127.0.0.1:36076_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.HdfsDirectoryFactory
   [junit4]   2> 26624 INFO  (coreLoadExecutor-71-thread-1-processing-n:127.0.0.1:36076_) [n:127.0.0.1:36076_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:33325/solr_hdfs_home
   [junit4]   2> 26624 INFO  (coreLoadExecutor-71-thread-1-processing-n:127.0.0.1:36076_) [n:127.0.0.1:36076_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
   [junit4]   2> 26624 INFO  (coreLoadExecutor-71-thread-1-processing-n:127.0.0.1:36076_) [n:127.0.0.1:36076_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_1A7B2F875163F270-001/control-001/cores/collection1], dataDir=[hdfs://localhost:33325/solr_hdfs_home/control_collection/core_node1/data/]
   [junit4]   2> 26624 INFO  (coreLoadExecutor-71-thread-1-processing-n:127.0.0.1:36076_) [n:127.0.0.1:36076_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@2039e2dd
   [junit4]   2> 26624 INFO  (coreLoadExecutor-71-thread-1-processing-n:127.0.0.1:36076_) [n:127.0.0.1:36076_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:33325/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 26666 INFO  (coreLoadExecutor-71-thread-1-processing-n:127.0.0.1:36076_) [n:127.0.0.1:36076_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:33325/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 26671 INFO  (coreLoadExecutor-71-thread-1-processing-n:127.0.0.1:36076_) [n:127.0.0.1:36076_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=hdfs://localhost:33325/solr_hdfs_home/control_collection/core_node1/data/index/
   [junit4]   2> 26692 WARN  (coreLoadExecutor-71-thread-1-processing-n:127.0.0.1:36076_) [n:127.0.0.1:36076_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory 'hdfs:/localhost:33325/solr_hdfs_home/control_collection/core_node1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 26692 INFO  (coreLoadExecutor-71-thread-1-processing-n:127.0.0.1:36076_) [n:127.0.0.1:36076_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:33325/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 26732 INFO  (coreLoadExecutor-71-thread-1-processing-n:127.0.0.1:36076_) [n:127.0.0.1:36076_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:33325/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 26732 INFO  (coreLoadExecutor-71-thread-1-processing-n:127.0.0.1:36076_) [n:127.0.0.1:36076_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=44, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.6527460698289071]
   [junit4]   2> 26952 INFO  (IPC Server handler 3 on 33325) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:41506 is added to blk_1073741825_1001{UCState=COMMITTED, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-c6bf4594-378f-46e4-82d1-03a82587dc80:NORMAL:127.0.0.1:41506|RBW], ReplicaUC[[DISK]DS-82c5b856-f875-4878-88de-1da30e9cb5e2:NORMAL:127.0.0.1:59002|RBW]]} size 71
   [junit4]   2> 26953 INFO  (IPC Server handler 4 on 33325) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:59002 is added to blk_1073741825_1001 size 71
   [junit4]   2> 27362 INFO  (coreLoadExecutor-71-thread-1-processing-n:127.0.0.1:36076_) [n:127.0.0.1:36076_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(HdfsDirectory@hdfs://localhost:33325/solr_hdfs_home/control_collection/core_node1/data/index lockFactory=org.apache.solr.store.hdfs.HdfsLockFactory@f096322; maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 27362 INFO  (coreLoadExecutor-71-thread-1-processing-n:127.0.0.1:36076_) [n:127.0.0.1:36076_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 27376 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:36076_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in hdfs://localhost:33325/solr_hdfs_home/control_collection/core_node1/data/
   [junit4]   2> 27380 INFO  (coreLoadExecutor-71-thread-1-processing-n:127.0.0.1:36076_) [n:127.0.0.1:36076_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 27380 INFO  (coreLoadExecutor-71-thread-1-processing-n:127.0.0.1:36076_) [n:127.0.0.1:36076_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 27380 INFO  (coreLoadExecutor-71-thread-1-processing-n:127.0.0.1:36076_) [n:127.0.0.1:36076_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 27380 INFO  (coreLoadExecutor-71-thread-1-processing-n:127.0.0.1:36076_) [n:127.0.0.1:36076_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 27381 INFO  (coreLoadExecutor-71-thread-1-processing-n:127.0.0.1:36076_) [n:127.0.0.1:36076_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 27381 INFO  (coreLoadExecutor-71-thread-1-processing-n:127.0.0.1:36076_) [n:127.0.0.1:36076_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 27381 INFO  (coreLoadExecutor-71-thread-1-processing-n:127.0.0.1:36076_) [n:127.0.0.1:36076_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 27381 INFO  (coreLoadExecutor-71-thread-1-processing-n:127.0.0.1:36076_) [n:127.0.0.1:36076_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 27381 INFO  (coreLoadExecutor-71-thread-1-processing-n:127.0.0.1:36076_) [n:127.0.0.1:36076_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 27381 WARN  (coreLoadExecutor-71-thread-1-processing-n:127.0.0.1:36076_) [n:127.0.0.1:36076_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
   [junit4]   2> 27383 INFO  (coreLoadExecutor-71-thread-1-processing-n:127.0.0.1:36076_) [n:127.0.0.1:36076_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 27383 INFO  (coreLoadExecutor-71-thread-1-processing-n:127.0.0.1:36076_) [n:127.0.0.1:36076_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 27384 INFO  (coreLoadExecutor-71-thread-1-processing-n:127.0.0.1:36076_) [n:127.0.0.1:36076_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 27385 INFO  (coreLoadExecutor-71-thread-1-processing-n:127.0.0.1:36076_) [n:127.0.0.1:36076_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 27389 INFO  (coreLoadExecutor-71-thread-1-processing-n:127.0.0.1:36076_) [n:127.0.0.1:36076_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/mbeans,standard,/update/json/docs,/admin/luke,/export,/get,/admin/properties,/update/json,/admin/threads,/dump,/update/csv,/sql,/graph,/admin/segments,/admin/system,/replication,/config,/stream,/schema,/admin/plugins,/admin/logging,/admin/ping,/update,/admin/file,/terms
   [junit4]   2> 27389 INFO  (coreLoadExecutor-71-thread-1-processing-n:127.0.0.1:36076_) [n:127.0.0.1:36076_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 27389 INFO  (coreLoadExecutor-71-thread-1-processing-n:127.0.0.1:36076_) [n:127.0.0.1:36076_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
   [junit4]   2> 27389 INFO  (coreLoadExecutor-71-thread-1-processing-n:127.0.0.1:36076_) [n:127.0.0.1:36076_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 27389 INFO  (coreLoadExecutor-71-thread-1-processing-n:127.0.0.1:36076_) [n:127.0.0.1:36076_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=2
   [junit4]   2> 27407 INFO  (coreLoadExecutor-71-thread-1-processing-n:127.0.0.1:36076_) [n:127.0.0.1:36076_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 27407 INFO  (coreLoadExecutor-71-thread-1-processing-n:127.0.0.1:36076_) [n:127.0.0.1:36076_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 27409 INFO  (coreLoadExecutor-71-thread-1-processing-n:127.0.0.1:36076_) [n:127.0.0.1:36076_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=47, maxMergeAtOnceExplicit=10, maxMergedSegmentMB=64.4306640625, floorSegmentMB=1.060546875, forceMergeDeletesPctAllowed=24.064354660959182, segmentsPerTier=17.0, maxCFSSegmentSizeMB=0.275390625, noCFSRatio=0.865211559398727
   [junit4]   2> 27467 INFO  (coreLoadExecutor-71-thread-1-processing-n:127.0.0.1:36076_) [n:127.0.0.1:36076_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(HdfsDirectory@hdfs://localhost:33325/solr_hdfs_home/control_collection/core_node1/data/index lockFactory=org.apache.solr.store.hdfs.HdfsLockFactory@f096322; maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 27467 INFO  (coreLoadExecutor-71-thread-1-processing-n:127.0.0.1:36076_) [n:127.0.0.1:36076_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 27467 INFO  (coreLoadExecutor-71-thread-1-processing-n:127.0.0.1:36076_) [n:127.0.0.1:36076_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@79accf24[collection1] main]
   [junit4]   2> 27468 INFO  (coreLoadExecutor-71-thread-1-processing-n:127.0.0.1:36076_) [n:127.0.0.1:36076_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/control_collection]
   [junit4]   2> 27469 INFO  (coreLoadExecutor-71-thread-1-processing-n:127.0.0.1:36076_) [n:127.0.0.1:36076_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/control_collection] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 27469 INFO  (coreLoadExecutor-71-thread-1-processing-n:127.0.0.1:36076_) [n:127.0.0.1:36076_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 27469 INFO  (coreLoadExecutor-71-thread-1-processing-n:127.0.0.1:36076_) [n:127.0.0.1:36076_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 27469 INFO  (coreLoadExecutor-71-thread-1-processing-n:127.0.0.1:36076_) [n:127.0.0.1:36076_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 27469 INFO  (coreLoadExecutor-71-thread-1-processing-n:127.0.0.1:36076_) [n:127.0.0.1:36076_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 27470 INFO  (coreLoadExecutor-71-thread-1-processing-n:127.0.0.1:36076_) [n:127.0.0.1:36076_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 27470 INFO  (coreLoadExecutor-71-thread-1-processing-n:127.0.0.1:36076_) [n:127.0.0.1:36076_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 27470 INFO  (coreLoadExecutor-71-thread-1-processing-n:127.0.0.1:36076_) [n:127.0.0.1:36076_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 27473 INFO  (coreLoadExecutor-71-thread-1-processing-n:127.0.0.1:36076_) [n:127.0.0.1:36076_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 27474 INFO  (searcherExecutor-72-thread-1-processing-n:127.0.0.1:36076_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:36076_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@79accf24[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 27475 INFO  (coreLoadExecutor-71-thread-1-processing-n:127.0.0.1:36076_) [n:127.0.0.1:36076_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 27475 INFO  (coreLoadExecutor-71-thread-1-processing-n:127.0.0.1:36076_) [n:127.0.0.1:36076_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 27475 INFO  (coreLoadExecutor-71-thread-1-processing-n:127.0.0.1:36076_) [n:127.0.0.1:36076_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 27475 INFO  (coreLoadExecutor-71-thread-1-processing-n:127.0.0.1:36076_) [n:127.0.0.1:36076_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1541067622733316096
   [junit4]   2> 27477 INFO  (coreLoadExecutor-71-thread-1-processing-n:127.0.0.1:36076_) [n:127.0.0.1:36076_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 2.0ms to seed version buckets with highest version 1541067622733316096
   [junit4]   2> 27478 INFO  (coreLoadExecutor-71-thread-1-processing-n:127.0.0.1:36076_) [n:127.0.0.1:36076_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 27478 INFO  (coreLoadExecutor-71-thread-1-processing-n:127.0.0.1:36076_) [n:127.0.0.1:36076_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 27479 INFO  (coreZkRegister-65-thread-1-processing-n:127.0.0.1:36076_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:36076_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:36076 collection:control_collection shard:shard1
   [junit4]   2> 27480 INFO  (coreZkRegister-65-thread-1-processing-n:127.0.0.1:36076_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:36076_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 27482 INFO  (coreZkRegister-65-thread-1-processing-n:127.0.0.1:36076_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:36076_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/96316726256009222-core_node1-n_0000000000
   [junit4]   2> 27483 INFO  (coreZkRegister-65-thread-1-processing-n:127.0.0.1:36076_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:36076_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard=shard1 and weAreReplacement=false and leaderVoteWait=10000
   [junit4]   2> 27484 INFO  (coreZkRegister-65-thread-1-processing-n:127.0.0.1:36076_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:36076_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 27484 INFO  (coreZkRegister-65-thread-1-processing-n:127.0.0.1:36076_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:36076_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 27484 INFO  (coreZkRegister-65-thread-1-processing-n:127.0.0.1:36076_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:36076_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:36076/collection1/
   [junit4]   2> 27484 INFO  (coreZkRegister-65-thread-1-processing-n:127.0.0.1:36076_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:36076_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 27484 INFO  (coreZkRegister-65-thread-1-processing-n:127.0.0.1:36076_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:36076_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:36076/collection1/ has no replicas
   [junit4]   2> 27485 INFO  (OverseerStateUpdate-96316726256009222-127.0.0.1:36076_-n_0000000000) [n:127.0.0.1:36076_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 27485 INFO  (coreZkRegister-65-thread-1-processing-n:127.0.0.1:36076_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:36076_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 27486 INFO  (zkCallback-32-thread-2-processing-n:127.0.0.1:36076_) [n:127.0.0.1:36076_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 27488 INFO  (coreZkRegister-65-thread-1-processing-n:127.0.0.1:36076_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:36076_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/control_collection/leaders/shard1/leader after winning as /collections/control_collection/leader_elect/shard1/election/96316726256009222-core_node1-n_0000000000
   [junit4]   2> 27490 INFO  (coreZkRegister-65-thread-1-processing-n:127.0.0.1:36076_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:36076_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:36076/collection1/ shard1
   [junit4]   2> 27490 INFO  (OverseerStateUpdate-96316726256009222-127.0.0.1:36076_-n_0000000000) [n:127.0.0.1:36076_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:36076",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 2
   [junit4]   2> 27592 INFO  (zkCallback-32-thread-2-processing-n:127.0.0.1:36076_) [n:127.0.0.1:36076_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 27641 INFO  (coreZkRegister-65-thread-1-processing-n:127.0.0.1:36076_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:36076_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:36076/collection1/ and leader is http://127.0.0.1:36076/collection1/
   [junit4]   2> 27641 INFO  (coreZkRegister-65-thread-1-processing-n:127.0.0.1:36076_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:36076_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:36076
   [junit4]   2> 27641 INFO  (coreZkRegister-65-thread-1-processing-n:127.0.0.1:36076_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:36076_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 27641 INFO  (coreZkRegister-65-thread-1-processing-n:127.0.0.1:36076_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:36076_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 27641 INFO  (coreZkRegister-65-thread-1-processing-n:127.0.0.1:36076_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:36076_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 27642 INFO  (coreZkRegister-65-thread-1-processing-n:127.0.0.1:36076_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:36076_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Checking legacy cluster state for collection control_collection
   [junit4]   2> 27646 INFO  (OverseerStateUpdate-96316726256009222-127.0.0.1:36076_-n_0000000000) [n:127.0.0.1:36076_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "dataDir":"hdfs://localhost:33325/solr_hdfs_home/control_collection/core_node1/data/",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:36076",
   [junit4]   2>   "node_name":"127.0.0.1:36076_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "ulogDir":"hdfs://localhost:33325/solr_hdfs_home/control_collection/core_node1/data/tlog"} current state version: 3
   [junit4]   2> 27646 INFO  (OverseerStateUpdate-96316726256009222-127.0.0.1:36076_-n_0000000000) [n:127.0.0.1:36076_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "dataDir":"hdfs://localhost:33325/solr_hdfs_home/control_collection/core_node1/data/",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:36076",
   [junit4]   2>   "node_name":"127.0.0.1:36076_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "ulogDir":"hdfs://localhost:33325/solr_hdfs_home/control_collection/core_node1/data/tlog"}
   [junit4]   2> 27751 INFO  (zkCallback-32-thread-1-processing-n:127.0.0.1:36076_) [n:127.0.0.1:36076_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 27908 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 27909 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 27911 INFO  (zkCallback-36-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3b538ccc name:ZooKeeperConnection Watcher:127.0.0.1:54319/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 27911 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 27911 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 27911 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 27913 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [    ] o.a.s.c.c.ZkStateReader Loaded empty cluster properties
   [junit4]   2> 27916 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 27917 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[1A7B2F875163F270]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 27988 INFO  (TEST-HdfsCollectionsAPIDistributedZk

[...truncated too long message...]

t.doRequest(LBHttpSolrClient.java:403)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:356)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1291)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1061)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:997)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1599)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1620)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:987)Throwable #5: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=13298, name=collection1, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    > Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:50325: collection already exists: awholynewstresscollection_collection1_3
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([1A7B2F875163F270]:0)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:590)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:259)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:248)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:403)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:356)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1291)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1061)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:997)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1599)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1620)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:987)Throwable #6: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=13299, name=collection2, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    > Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:50325: collection already exists: awholynewstresscollection_collection2_3
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([1A7B2F875163F270]:0)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:590)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:259)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:248)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:403)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:356)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1291)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1061)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:997)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1599)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1620)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:987)Throwable #7: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=13302, name=collection5, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    > Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:53671: collection already exists: awholynewstresscollection_collection5_4
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([1A7B2F875163F270]:0)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:590)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:259)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:248)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:403)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:356)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1291)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1061)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:997)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1599)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1620)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:987)
   [junit4]   2> 2101743 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[1A7B2F875163F270]-worker) [    ] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_1A7B2F875163F270-001
   [junit4]   2> Jul 28, 2016 4:04:45 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene62), sim=ClassicSimilarity, locale=it-IT, timezone=Europe/Warsaw
   [junit4]   2> NOTE: Linux 3.13.0-85-generic amd64/Oracle Corporation 1.8.0_74 (64-bit)/cpus=4,threads=1,free=162130120,total=477626368
   [junit4]   2> NOTE: All tests run in this JVM: [TestWriterPerf, TestCloudPseudoReturnFields, TestCollationField, HdfsUnloadDistributedZkTest, FullHLLTest, StatsComponentTest, TestOrdValues, TestRequestStatusCollectionAPI, CurrencyFieldXmlFileTest, SearchHandlerTest, TestDocBasedVersionConstraints, TestRecovery, TestCloudSchemaless, RequestHandlersTest, StatelessScriptUpdateProcessorFactoryTest, TestCloudDeleteByQuery, TestDefaultStatsCache, TestSolrCLIRunExample, TestExtendedDismaxParser, TestReversedWildcardFilterFactory, TestSolrJ, TestZkChroot, DeleteInactiveReplicaTest, AutoCommitTest, TestCSVResponseWriter, SharedFSAutoReplicaFailoverUtilsTest, TestHdfsCloudBackupRestore, TestXmlQParserPlugin, DebugComponentTest, BlockCacheTest, SliceStateTest, SmileWriterTest, TestOnReconnectListenerSupport, TestMacros, SolrCoreCheckLockOnStartupTest, TestRebalanceLeaders, TestFieldCacheSort, HighlighterConfigTest, DocValuesMultiTest, PKIAuthenticationIntegrationTest, SparseHLLTest, BlockJoinFacetDistribTest, FullSolrCloudDistribCmdsTest, TestSQLHandler, RegexBoostProcessorTest, FieldMutatingUpdateProcessorTest, TestPerFieldSimilarity, TestFieldCache, TestSolrDeletionPolicy1, TestDistributedMissingSort, TestReload, TestCollapseQParserPlugin, VersionInfoTest, FieldAnalysisRequestHandlerTest, TlogReplayBufferedWhileIndexingTest, TestClassicSimilarityFactory, AnalysisErrorHandlingTest, SampleTest, SaslZkACLProviderTest, QueryParsingTest, TestPerFieldSimilarityClassic, DirectSolrConnectionTest, TestSuggestSpellingConverter, TestBulkSchemaAPI, UnloadDistributedZkTest, TestFieldSortValues, WrapperMergePolicyFactoryTest, TestTolerantSearch, TestUtils, CollectionsAPISolrJTest, TestFieldCollectionResource, BooleanFieldTest, TestManagedResource, UpdateParamsTest, TestStressReorder, BadIndexSchemaTest, CursorMarkTest, TestDefaultSearchFieldResource, PluginInfoTest, TestFieldTypeResource, TestNamedUpdateProcessors, OverriddenZkACLAndCredentialsProvidersTest, DistributedFacetPivotLongTailTest, MigrateRouteKeyTest, TestUseDocValuesAsStored2, TestQueryWrapperFilter, ConnectionManagerTest, TestStressUserVersions, CleanupOldIndexTest, TestFastWriter, HdfsSyncSliceTest, TestCharFilters, TestSweetSpotSimilarityFactory, CoreSorterTest, TestStressRecovery, DirectUpdateHandlerTest, ScriptEngineTest, DirectUpdateHandlerOptimizeTest, TestGraphMLResponseWriter, TestSchemaSimilarityResource, LeaderFailoverAfterPartitionTest, BasicDistributedZk2Test, CollectionsAPIDistributedZkTest]
   [junit4] Completed [301/624 (2!)] on J2 in 491.88s, 1 test, 1 error <<< FAILURES!

[...truncated 61110 lines...]



[JENKINS] Lucene-Solr-NightlyTests-6.x - Build # 127 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-6.x/127/

2 tests failed.
FAILED:  org.apache.solr.cloud.LeaderFailoverAfterPartitionTest.test

Error Message:
No live SolrServers available to handle this request:[http://127.0.0.1:38909/c8n_1x3_lf_shard1_replica2]

Stack Trace:
org.apache.solr.client.solrj.impl.CloudSolrClient$RouteException: No live SolrServers available to handle this request:[http://127.0.0.1:38909/c8n_1x3_lf_shard1_replica2]
	at __randomizedtesting.SeedInfo.seed([10ABF5368D693666:98FFCAEC23955B9E]:0)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.directUpdate(CloudSolrClient.java:760)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1172)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1061)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:997)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
	at org.apache.solr.cloud.HttpPartitionTest.sendDoc(HttpPartitionTest.java:592)
	at org.apache.solr.cloud.HttpPartitionTest.sendDoc(HttpPartitionTest.java:578)
	at org.apache.solr.cloud.LeaderFailoverAfterPartitionTest.testRf3WithLeaderFailover(LeaderFailoverAfterPartitionTest.java:174)
	at org.apache.solr.cloud.LeaderFailoverAfterPartitionTest.test(LeaderFailoverAfterPartitionTest.java:55)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:38909/c8n_1x3_lf_shard1_replica2]
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:383)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.lambda$directUpdate$0(CloudSolrClient.java:735)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	... 1 more
Caused by: org.apache.solr.client.solrj.SolrServerException: Server refused connection at: http://127.0.0.1:38909/c8n_1x3_lf_shard1_replica2
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:597)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:259)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:248)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:403)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:356)
	... 6 more
Caused by: java.net.ConnectException: Connection refused
	at java.net.PlainSocketImpl.socketConnect(Native Method)
	at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
	at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
	at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
	at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
	at java.net.Socket.connect(Socket.java:589)
	at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:117)
	at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:177)
	at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:304)
	at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:611)
	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:446)
	at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:495)
	... 10 more


FAILED:  org.apache.solr.handler.TestReqParamsAPI.test

Error Message:
Could not get expected value  'first' for path 'response/params/x/_appends_/add' full output: {   "responseHeader":{     "status":0,     "QTime":0},   "response":{     "znodeVersion":3,     "params":{       "x":{         "a":"A val",         "b":"B val",         "":{"v":0}},       "y":{         "p":"P val",         "q":"Q val",         "":{"v":2}}}}},  from server:  https://127.0.0.1:54160/hz_/w/collection1

Stack Trace:
java.lang.AssertionError: Could not get expected value  'first' for path 'response/params/x/_appends_/add' full output: {
  "responseHeader":{
    "status":0,
    "QTime":0},
  "response":{
    "znodeVersion":3,
    "params":{
      "x":{
        "a":"A val",
        "b":"B val",
        "":{"v":0}},
      "y":{
        "p":"P val",
        "q":"Q val",
        "":{"v":2}}}}},  from server:  https://127.0.0.1:54160/hz_/w/collection1
	at __randomizedtesting.SeedInfo.seed([10ABF5368D693666:98FFCAEC23955B9E]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.apache.solr.core.TestSolrConfigHandler.testForResponseElement(TestSolrConfigHandler.java:481)
	at org.apache.solr.handler.TestReqParamsAPI.testReqParams(TestReqParamsAPI.java:230)
	at org.apache.solr.handler.TestReqParamsAPI.test(TestReqParamsAPI.java:61)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 11289 lines...]
   [junit4] Suite: org.apache.solr.handler.TestReqParamsAPI
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReqParamsAPI_10ABF5368D693666-001/init-core-data-001
   [junit4]   2> 176609 INFO  (SUITE-TestReqParamsAPI-seed#[10ABF5368D693666]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
   [junit4]   2> 176609 INFO  (SUITE-TestReqParamsAPI-seed#[10ABF5368D693666]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /hz_/w
   [junit4]   2> 176614 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 176614 INFO  (Thread-393) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 176615 INFO  (Thread-393) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 176714 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [    ] o.a.s.c.ZkTestServer start zk server on port:46324
   [junit4]   2> 176715 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 176715 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 176722 INFO  (zkCallback-239-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1667d566 name:ZooKeeperConnection Watcher:127.0.0.1:46324 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 176722 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 176722 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 176722 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 176725 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 176725 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 176727 INFO  (zkCallback-240-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@44e927e3 name:ZooKeeperConnection Watcher:127.0.0.1:46324/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 176727 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 176727 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 176727 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 176728 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 176730 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 176731 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 176732 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 176732 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 176735 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 176735 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 176736 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 176737 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 176738 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 176738 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 176740 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 176740 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 176741 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 176741 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 176743 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 176743 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 176744 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 176745 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 176746 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 176746 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 176747 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 176748 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 176751 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 176751 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 176753 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 176761 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 176761 INFO  (zkCallback-242-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2dd888a7 name:ZooKeeperConnection Watcher:127.0.0.1:46324/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 176761 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 176761 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 176876 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReqParamsAPI_10ABF5368D693666-001/control-001/cores/collection1
   [junit4]   2> 176879 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [    ] o.e.j.s.Server jetty-9.3.8.v20160314
   [junit4]   2> 176881 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@340233bd{/hz_/w,null,AVAILABLE}
   [junit4]   2> 176884 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [    ] o.e.j.s.ServerConnector Started ServerConnector@1d45c581{SSL,[ssl, http/1.1]}{127.0.0.1:56468}
   [junit4]   2> 176884 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [    ] o.e.j.s.Server Started @180009ms
   [junit4]   2> 176884 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReqParamsAPI_10ABF5368D693666-001/tempDir-001/control/data, hostContext=/hz_/w, hostPort=56468, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReqParamsAPI_10ABF5368D693666-001/control-001/cores}
   [junit4]   2> 176885 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@73d16e93
   [junit4]   2> 176885 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReqParamsAPI_10ABF5368D693666-001/control-001'
   [junit4]   2> 176885 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 176885 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 176885 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 176890 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 176890 INFO  (zkCallback-243-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@a2077df name:ZooKeeperConnection Watcher:127.0.0.1:46324/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 176890 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 176891 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 176892 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 176892 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReqParamsAPI_10ABF5368D693666-001/control-001/solr.xml
   [junit4]   2> 176897 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReqParamsAPI_10ABF5368D693666-001/control-001/cores
   [junit4]   2> 176897 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [    ] o.a.s.c.CoreContainer New CoreContainer 1227848967
   [junit4]   2> 176897 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReqParamsAPI_10ABF5368D693666-001/control-001]
   [junit4]   2> 176898 WARN  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReqParamsAPI_10ABF5368D693666-001/control-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReqParamsAPI_10ABF5368D693666-001/control-001/lib
   [junit4]   2> 176898 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,connectionsEvictorSleepDelay : 5000,maxConnectionIdleTime : 40000,
   [junit4]   2> 176899 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 176900 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 176900 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 176900 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46324/solr
   [junit4]   2> 176900 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 176900 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 176901 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 176902 INFO  (zkCallback-246-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3de46ce9 name:ZooKeeperConnection Watcher:127.0.0.1:46324 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 176903 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 176903 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 176904 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [n:127.0.0.1:56468_hz_%2Fw    ] o.a.s.c.ZkController Added new OnReconnect listener org.apache.solr.cloud.ZkController$$Lambda$50/377473286@34b08d43
   [junit4]   2> 176905 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [n:127.0.0.1:56468_hz_%2Fw    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 176906 INFO  (zkCallback-247-thread-1-processing-n:127.0.0.1:56468_hz_%2Fw) [n:127.0.0.1:56468_hz_%2Fw    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@26e24523 name:ZooKeeperConnection Watcher:127.0.0.1:46324/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 176908 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [n:127.0.0.1:56468_hz_%2Fw    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 176908 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [n:127.0.0.1:56468_hz_%2Fw    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 176910 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [n:127.0.0.1:56468_hz_%2Fw    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 176912 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [n:127.0.0.1:56468_hz_%2Fw    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 176914 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [n:127.0.0.1:56468_hz_%2Fw    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 176916 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [n:127.0.0.1:56468_hz_%2Fw    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 176917 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [n:127.0.0.1:56468_hz_%2Fw    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 176918 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [n:127.0.0.1:56468_hz_%2Fw    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 176919 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [n:127.0.0.1:56468_hz_%2Fw    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 176920 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [n:127.0.0.1:56468_hz_%2Fw    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 176921 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [n:127.0.0.1:56468_hz_%2Fw    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 176921 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [n:127.0.0.1:56468_hz_%2Fw    ] o.a.s.c.c.ZkStateReader Loaded cluster properties: {urlScheme=https}
   [junit4]   2> 176922 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [n:127.0.0.1:56468_hz_%2Fw    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (0)
   [junit4]   2> 176924 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [n:127.0.0.1:56468_hz_%2Fw    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 176925 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [n:127.0.0.1:56468_hz_%2Fw    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 176925 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [n:127.0.0.1:56468_hz_%2Fw    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 176926 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [n:127.0.0.1:56468_hz_%2Fw    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/96311457290387461-127.0.0.1:56468_hz_%2Fw-n_0000000000
   [junit4]   2> 176926 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [n:127.0.0.1:56468_hz_%2Fw    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:56468_hz_%2Fw
   [junit4]   2> 176926 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [n:127.0.0.1:56468_hz_%2Fw    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 176927 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [n:127.0.0.1:56468_hz_%2Fw    ] o.a.s.c.Overseer Overseer (id=96311457290387461-127.0.0.1:56468_hz_%2Fw-n_0000000000) starting
   [junit4]   2> 176928 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [n:127.0.0.1:56468_hz_%2Fw    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 176933 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [n:127.0.0.1:56468_hz_%2Fw    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 176933 INFO  (OverseerCollectionConfigSetProcessor-96311457290387461-127.0.0.1:56468_hz_%2Fw-n_0000000000) [n:127.0.0.1:56468_hz_%2Fw    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 176933 INFO  (OverseerStateUpdate-96311457290387461-127.0.0.1:56468_hz_%2Fw-n_0000000000) [n:127.0.0.1:56468_hz_%2Fw    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 176934 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [n:127.0.0.1:56468_hz_%2Fw    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:56468_hz_%2Fw
   [junit4]   2> 176934 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [n:127.0.0.1:56468_hz_%2Fw    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:56468_hz_%2Fw
   [junit4]   2> 176936 INFO  (OverseerStateUpdate-96311457290387461-127.0.0.1:56468_hz_%2Fw-n_0000000000) [n:127.0.0.1:56468_hz_%2Fw    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (0)
   [junit4]   2> 176936 INFO  (zkCallback-247-thread-1-processing-n:127.0.0.1:56468_hz_%2Fw) [n:127.0.0.1:56468_hz_%2Fw    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [0])
   [junit4]   2> 176937 INFO  (zkCallback-247-thread-1-processing-n:127.0.0.1:56468_hz_%2Fw) [n:127.0.0.1:56468_hz_%2Fw    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 176962 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [n:127.0.0.1:56468_hz_%2Fw    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 176962 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [n:127.0.0.1:56468_hz_%2Fw    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 176963 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [n:127.0.0.1:56468_hz_%2Fw    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReqParamsAPI_10ABF5368D693666-001/control-001/cores
   [junit4]   2> 176964 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [n:127.0.0.1:56468_hz_%2Fw    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, loadOnStartup=true, configSetProperties=configsetprops.json, collection=control_collection, coreNodeName=, dataDir=data/, shard=}
   [junit4]   2> 176964 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [n:127.0.0.1:56468_hz_%2Fw    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReqParamsAPI_10ABF5368D693666-001/control-001/cores/collection1
   [junit4]   2> 176964 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [n:127.0.0.1:56468_hz_%2Fw    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 176965 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [n:127.0.0.1:56468_hz_%2Fw    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0
   [junit4]   2> 176965 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [n:127.0.0.1:56468_hz_%2Fw    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 176966 INFO  (coreLoadExecutor-467-thread-1-processing-n:127.0.0.1:56468_hz_%2Fw) [n:127.0.0.1:56468_hz_%2Fw c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 176966 INFO  (coreLoadExecutor-467-thread-1-processing-n:127.0.0.1:56468_hz_%2Fw) [n:127.0.0.1:56468_hz_%2Fw c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 176968 INFO  (coreLoadExecutor-467-thread-1-processing-n:127.0.0.1:56468_hz_%2Fw) [n:127.0.0.1:56468_hz_%2Fw c:control_collection   x:collection1] o.a.s.c.ZkController Collection control_collection not visible yet, but flagging it so a watch is registered when it becomes visible
   [junit4]   2> 176968 INFO  (OverseerStateUpdate-96311457290387461-127.0.0.1:56468_hz_%2Fw-n_0000000000) [n:127.0.0.1:56468_hz_%2Fw    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:56468/hz_/w",
   [junit4]   2>   "node_name":"127.0.0.1:56468_hz_%2Fw",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 176968 INFO  (OverseerStateUpdate-96311457290387461-127.0.0.1:56468_hz_%2Fw-n_0000000000) [n:127.0.0.1:56468_hz_%2Fw    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:56468/hz_/w",
   [junit4]   2>   "node_name":"127.0.0.1:56468_hz_%2Fw",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 176969 INFO  (OverseerStateUpdate-96311457290387461-127.0.0.1:56468_hz_%2Fw-n_0000000000) [n:127.0.0.1:56468_hz_%2Fw    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 176969 INFO  (coreLoadExecutor-467-thread-1-processing-n:127.0.0.1:56468_hz_%2Fw) [n:127.0.0.1:56468_hz_%2Fw c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Deleting data for [control_collection]
   [junit4]   2> 176970 INFO  (OverseerStateUpdate-96311457290387461-127.0.0.1:56468_hz_%2Fw-n_0000000000) [n:127.0.0.1:56468_hz_%2Fw    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 176971 INFO  (zkCallback-247-thread-1-processing-n:127.0.0.1:56468_hz_%2Fw) [n:127.0.0.1:56468_hz_%2Fw    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 176978 INFO  (coreLoadExecutor-467-thread-1-processing-n:127.0.0.1:56468_hz_%2Fw) [n:127.0.0.1:56468_hz_%2Fw c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 176978 INFO  (coreLoadExecutor-467-thread-1-processing-n:127.0.0.1:56468_hz_%2Fw) [n:127.0.0.1:56468_hz_%2Fw c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 176978 INFO  (coreLoadExecutor-467-thread-1-processing-n:127.0.0.1:56468_hz_%2Fw) [n:127.0.0.1:56468_hz_%2Fw c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 176979 INFO  (coreLoadExecutor-467-thread-1-processing-n:127.0.0.1:56468_hz_%2Fw) [n:127.0.0.1:56468_hz_%2Fw c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 176979 INFO  (coreLoadExecutor-467-thread-1-processing-n:127.0.0.1:56468_hz_%2Fw) [n:127.0.0.1:56468_hz_%2Fw c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/control_collection]
   [junit4]   2> 176979 INFO  (coreLoadExecutor-467-thread-1-processing-n:127.0.0.1:56468_hz_%2Fw) [n:127.0.0.1:56468_hz_%2Fw c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/control_collection] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 176979 INFO  (coreLoadExecutor-467-thread-1-processing-n:127.0.0.1:56468_hz_%2Fw) [n:127.0.0.1:56468_hz_%2Fw c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReqParamsAPI_10ABF5368D693666-001/control-001/cores/collection1'
   [junit4]   2> 176979 INFO  (coreLoadExecutor-467-thread-1-processing-n:127.0.0.1:56468_hz_%2Fw) [n:127.0.0.1:56468_hz_%2Fw c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 176980 INFO  (coreLoadExecutor-467-thread-1-processing-n:127.0.0.1:56468_hz_%2Fw) [n:127.0.0.1:56468_hz_%2Fw c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 176981 INFO  (coreLoadExecutor-467-thread-1-processing-n:127.0.0.1:56468_hz_%2Fw) [n:127.0.0.1:56468_hz_%2Fw c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 176987 INFO  (coreLoadExecutor-467-thread-1-processing-n:127.0.0.1:56468_hz_%2Fw) [n:127.0.0.1:56468_hz_%2Fw c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 176991 INFO  (coreLoadExecutor-467-thread-1-processing-n:127.0.0.1:56468_hz_%2Fw) [n:127.0.0.1:56468_hz_%2Fw c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.2.0
   [junit4]   2> 176999 INFO  (coreLoadExecutor-467-thread-1-processing-n:127.0.0.1:56468_hz_%2Fw) [n:127.0.0.1:56468_hz_%2Fw c:control_collection   x:collection1] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 177004 INFO  (coreLoadExecutor-467-thread-1-processing-n:127.0.0.1:56468_hz_%2Fw) [n:127.0.0.1:56468_hz_%2Fw c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 177136 INFO  (coreLoadExecutor-467-thread-1-processing-n:127.0.0.1:56468_hz_%2Fw) [n:127.0.0.1:56468_hz_%2Fw c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 177146 INFO  (coreLoadExecutor-467-thread-1-processing-n:127.0.0.1:56468_hz_%2Fw) [n:127.0.0.1:56468_hz_%2Fw c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 177147 INFO  (coreLoadExecutor-467-thread-1-processing-n:127.0.0.1:56468_hz_%2Fw) [n:127.0.0.1:56468_hz_%2Fw c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 177157 INFO  (coreLoadExecutor-467-thread-1-processing-n:127.0.0.1:56468_hz_%2Fw) [n:127.0.0.1:56468_hz_%2Fw c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 177159 INFO  (coreLoadExecutor-467-thread-1-processing-n:127.0.0.1:56468_hz_%2Fw) [n:127.0.0.1:56468_hz_%2Fw c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 177161 INFO  (coreLoadExecutor-467-thread-1-processing-n:127.0.0.1:56468_hz_%2Fw) [n:127.0.0.1:56468_hz_%2Fw c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 177162 INFO  (coreLoadExecutor-467-thread-1-processing-n:127.0.0.1:56468_hz_%2Fw) [n:127.0.0.1:56468_hz_%2Fw c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 177163 INFO  (coreLoadExecutor-467-thread-1-processing-n:127.0.0.1:56468_hz_%2Fw) [n:127.0.0.1:56468_hz_%2Fw c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0
   [junit4]   2> 177163 INFO  (coreLoadExecutor-467-thread-1-processing-n:127.0.0.1:56468_hz_%2Fw) [n:127.0.0.1:56468_hz_%2Fw c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 177164 INFO  (coreLoadExecutor-467-thread-1-processing-n:127.0.0.1:56468_hz_%2Fw) [n:127.0.0.1:56468_hz_%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 177164 INFO  (coreLoadExecutor-467-thread-1-processing-n:127.0.0.1:56468_hz_%2Fw) [n:127.0.0.1:56468_hz_%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReqParamsAPI_10ABF5368D693666-001/control-001/cores/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReqParamsAPI_10ABF5368D693666-001/control-001/cores/collection1/data/]
   [junit4]   2> 177164 INFO  (coreLoadExecutor-467-thread-1-processing-n:127.0.0.1:56468_hz_%2Fw) [n:127.0.0.1:56468_hz_%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@791c25b7
   [junit4]   2> 177165 INFO  (coreLoadExecutor-467-thread-1-processing-n:127.0.0.1:56468_hz_%2Fw) [n:127.0.0.1:56468_hz_%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReqParamsAPI_10ABF5368D693666-001/control-001/cores/collection1/data
   [junit4]   2> 177165 INFO  (coreLoadExecutor-467-thread-1-processing-n:127.0.0.1:56468_hz_%2Fw) [n:127.0.0.1:56468_hz_%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReqParamsAPI_10ABF5368D693666-001/control-001/cores/collection1/data/index/
   [junit4]   2> 177165 WARN  (coreLoadExecutor-467-thread-1-processing-n:127.0.0.1:56468_hz_%2Fw) [n:127.0.0.1:56468_hz_%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReqParamsAPI_10ABF5368D693666-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 177166 INFO  (coreLoadExecutor-467-thread-1-processing-n:127.0.0.1:56468_hz_%2Fw) [n:127.0.0.1:56468_hz_%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReqParamsAPI_10ABF5368D693666-001/control-001/cores/collection1/data/index
   [junit4]   2> 177167 INFO  (coreLoadExecutor-467-thread-1-processing-n:127.0.0.1:56468_hz_%2Fw) [n:127.0.0.1:56468_hz_%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=17, maxMergeAtOnceExplicit=27, maxMergedSegmentMB=96.1171875, floorSegmentMB=1.619140625, forceMergeDeletesPctAllowed=16.92658774746169, segmentsPerTier=40.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.643981415252059
   [junit4]   2> 177169 INFO  (coreLoadExecutor-467-thread-1-processing-n:127.0.0.1:56468_hz_%2Fw) [n:127.0.0.1:56468_hz_%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@45f59248 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@70047083),segFN=segments_1,generation=1}
   [junit4]   2> 177169 INFO  (coreLoadExecutor-467-thread-1-processing-n:127.0.0.1:56468_hz_%2Fw) [n:127.0.0.1:56468_hz_%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 177170 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:56468_hz_%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReqParamsAPI_10ABF5368D693666-001/control-001/cores/collection1/data/
   [junit4]   2> 177173 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:56468_hz_%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReqParamsAPI_10ABF5368D693666-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 177173 INFO  (coreLoadExecutor-467-thread-1-processing-n:127.0.0.1:56468_hz_%2Fw) [n:127.0.0.1:56468_hz_%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 177174 INFO  (coreLoadExecutor-467-thread-1-processing-n:127.0.0.1:56468_hz_%2Fw) [n:127.0.0.1:56468_hz_%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 177174 INFO  (coreLoadExecutor-467-thread-1-processing-n:127.0.0.1:56468_hz_%2Fw) [n:127.0.0.1:56468_hz_%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 177174 INFO  (coreLoadExecutor-467-thread-1-processing-n:127.0.0.1:56468_hz_%2Fw) [n:127.0.0.1:56468_hz_%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 177174 INFO  (coreLoadExecutor-467-thread-1-processing-n:127.0.0.1:56468_hz_%2Fw) [n:127.0.0.1:56468_hz_%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 177174 INFO  (coreLoadExecutor-467-thread-1-processing-n:127.0.0.1:56468_hz_%2Fw) [n:127.0.0.1:56468_hz_%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 177174 INFO  (coreLoadExecutor-467-thread-1-processing-n:127.0.0.1:56468_hz_%2Fw) [n:127.0.0.1:56468_hz_%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 177175 INFO  (coreLoadExecutor-467-thread-1-processing-n:127.0.0.1:56468_hz_%2Fw) [n:127.0.0.1:56468_hz_%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 177175 INFO  (coreLoadExecutor-467-thread-1-processing-n:127.0.0.1:56468_hz_%2Fw) [n:127.0.0.1:56468_hz_%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 177175 WARN  (coreLoadExecutor-467-thread-1-processing-n:127.0.0.1:56468_hz_%2Fw) [n:127.0.0.1:56468_hz_%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
   [junit4]   2> 177187 INFO  (coreLoadExecutor-467-thread-1-processing-n:127.0.0.1:56468_hz_%2Fw) [n:127.0.0.1:56468_hz_%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 177188 INFO  (coreLoadExecutor-467-thread-1-processing-n:127.0.0.1:56468_hz_%2Fw) [n:127.0.0.1:56468_hz_%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 177189 INFO  (coreLoadExecutor-467-thread-1-processing-n:127.0.0.1:56468_hz_%2Fw) [n:127.0.0.1:56468_hz_%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 177192 INFO  (coreLoadExecutor-467-thread-1-processing-n:127.0.0.1:56468_hz_%2Fw) [n:127.0.0.1:56468_hz_%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 177196 INFO  (coreLoadExecutor-467-thread-1-processing-n:127.0.0.1:56468_hz_%2Fw) [n:127.0.0.1:56468_hz_%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/mbeans,standard,/update/json/docs,/admin/luke,/export,/get,/admin/properties,/update/json,/admin/threads,/dump,/update/csv,/sql,/graph,/admin/segments,/admin/system,/replication,/config,/stream,/schema,/admin/plugins,/admin/logging,/admin/ping,/update,/admin/file,/terms
   [junit4]   2> 177196 INFO  (coreLoadExecutor-467-thread-1-processing-n:127.0.0.1:56468_hz_%2Fw) [n:127.0.0.1:56468_hz_%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 177196 INFO  (coreLoadExecutor-467-thread-1-processing-n:127.0.0.1:56468_hz_%2Fw) [n:127.0.0.1:56468_hz_%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 177196 INFO  (coreLoadExecutor-467-thread-1-processing-n:127.0.0.1:56468_hz_%2Fw) [n:127.0.0.1:56468_hz_%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 177197 INFO  (coreLoadExecutor-467-thread-1-processing-n:127.0.0.1:56468_hz_%2Fw) [n:127.0.0.1:56468_hz_%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 177197 INFO  (coreLoadExecutor-467-thread-1-processing-n:127.0.0.1:56468_hz_%2Fw) [n:127.0.0.1:56468_hz_%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 177199 INFO  (coreLoadExecutor-467-thread-1-processing-n:127.0.0.1:56468_hz_%2Fw) [n:127.0.0.1:56468_hz_%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=40, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.2579475976111197]
   [junit4]   2> 177201 INFO  (coreLoadExecutor-467-thread-1-processing-n:127.0.0.1:56468_hz_%2Fw) [n:127.0.0.1:56468_hz_%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@45f59248 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@70047083),segFN=segments_1,generation=1}
   [junit4]   2> 177201 INFO  (coreLoadExecutor-467-thread-1-processing-n:127.0.0.1:56468_hz_%2Fw) [n:127.0.0.1:56468_hz_%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 177202 INFO  (coreLoadExecutor-467-thread-1-processing-n:127.0.0.1:56468_hz_%2Fw) [n:127.0.0.1:56468_hz_%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@762e477b[collection1] main]
   [junit4]   2> 177202 INFO  (coreLoadExecutor-467-thread-1-processing-n:127.0.0.1:56468_hz_%2Fw) [n:127.0.0.1:56468_hz_%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/control_collection]
   [junit4]   2> 177205 INFO  (coreLoadExecutor-467-thread-1-processing-n:127.0.0.1:56468_hz_%2Fw) [n:127.0.0.1:56468_hz_%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/control_collection] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 177205 INFO  (coreLoadExecutor-467-thread-1-processing-n:127.0.0.1:56468_hz_%2Fw) [n:127.0.0.1:56468_hz_%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 177205 INFO  (coreLoadExecutor-467-thread-1-processing-n:127.0.0.1:56468_hz_%2Fw) [n:127.0.0.1:56468_hz_%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 177205 INFO  (coreLoadExecutor-467-thread-1-processing-n:127.0.0.1:56468_hz_%2Fw) [n:127.0.0.1:56468_hz_%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 177205 INFO  (coreLoadExecutor-467-thread-1-processing-n:127.0.0.1:56468_hz_%2Fw) [n:127.0.0.1:56468_hz_%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 177206 INFO  (coreLoadExecutor-467-thread-1-processing-n:127.0.0.1:56468_hz_%2Fw) [n:127.0.0.1:56468_hz_%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 177206 INFO  (coreLoadExecutor-467-thread-1-processing-n:127.0.0.1:56468_hz_%2Fw) [n:127.0.0.1:56468_hz_%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 177206 INFO  (coreLoadExecutor-467-thread-1-processing-n:127.0.0.1:56468_hz_%2Fw) [n:127.0.0.1:56468_hz_%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 177206 INFO  (coreLoadExecutor-467-thread-1-processing-n:127.0.0.1:56468_hz_%2Fw) [n:127.0.0.1:56468_hz_%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 177206 INFO  (coreLoadExecutor-467-thread-1-processing-n:127.0.0.1:56468_hz_%2Fw) [n:127.0.0.1:56468_hz_%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 177206 INFO  (coreLoadExecutor-467-thread-1-processing-n:127.0.0.1:56468_hz_%2Fw) [n:127.0.0.1:56468_hz_%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 177207 INFO  (coreLoadExecutor-467-thread-1-processing-n:127.0.0.1:56468_hz_%2Fw) [n:127.0.0.1:56468_hz_%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 177207 INFO  (coreLoadExecutor-467-thread-1-processing-n:127.0.0.1:56468_hz_%2Fw) [n:127.0.0.1:56468_hz_%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1540983317265907712
   [junit4]   2> 177208 INFO  (coreLoadExecutor-467-thread-1-processing-n:127.0.0.1:56468_hz_%2Fw) [n:127.0.0.1:56468_hz_%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 1.0ms to seed version buckets with highest version 1540983317265907712
   [junit4]   2> 177209 INFO  (coreLoadExecutor-467-thread-1-processing-n:127.0.0.1:56468_hz_%2Fw) [n:127.0.0.1:56468_hz_%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 177210 INFO  (searcherExecutor-468-thread-1-processing-n:127.0.0.1:56468_hz_%2Fw x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:56468_hz_%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@762e477b[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 177210 INFO  (coreLoadExecutor-467-thread-1-processing-n:127.0.0.1:56468_hz_%2Fw) [n:127.0.0.1:56468_hz_%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 177210 INFO  (coreZkRegister-461-thread-1-processing-n:127.0.0.1:56468_hz_%2Fw x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:56468_hz_%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:https://127.0.0.1:56468/hz_/w collection:control_collection shard:shard1
   [junit4]   2> 177211 INFO  (coreZkRegister-461-thread-1-processing-n:127.0.0.1:56468_hz_%2Fw x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:56468_hz_%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 177219 INFO  (coreZkRegister-461-thread-1-processing-n:127.0.0.1:56468_hz_%2Fw x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:56468_hz_%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/96311457290387461-core_node1-n_0000000000
   [junit4]   2> 177219 INFO  (coreZkRegister-461-thread-1-processing-n:127.0.0.1:56468_hz_%2Fw x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:56468_hz_%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard=shard1 and weAreReplacement=false and leaderVoteWait=10000
   [junit4]   2> 177221 INFO  (coreZkRegister-461-thread-1-processing-n:127.0.0.1:56468_hz_%2Fw x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:56468_hz_%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 177221 INFO  (coreZkRegister-461-thread-1-processing-n:127.0.0.1:56468_hz_%2Fw x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:56468_hz_%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 177221 INFO  (coreZkRegister-461-thread-1-processing-n:127.0.0.1:56468_hz_%2Fw x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:56468_hz_%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:56468/hz_/w/collection1/
   [junit4]   2> 177221 INFO  (coreZkRegister-461-thread-1-processing-n:127.0.0.1:56468_hz_%2Fw x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:56468_hz_%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 177222 INFO  (coreZkRegister-461-thread-1-processing-n:127.0.0.1:56468_hz_%2Fw x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:56468_hz_%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy https://127.0.0.1:56468/hz_/w/collection1/ has no replicas
   [junit4]   2> 177222 INFO  (OverseerStateUpdate-96311457290387461-127.0.0.1:56468_hz_%2Fw-n_0000000000) [n:127.0.0.1:56468_hz_%2Fw    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 177223 INFO  (coreZkRegister-461-thread-1-processing-n:127.0.0.1:56468_hz_%2Fw x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:56468_hz_%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 177225 INFO  (coreZkRegister-461-thread-1-processing-n:127.0.0.1:56468_hz_%2Fw x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:56468_hz_%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/control_collection/leaders/shard1/leader after winning as /collections/control_collection/leader_elect/shard1/election/96311457290387461-core_node1-n_0000000000
   [junit4]   2> 177227 INFO  (coreZkRegister-461-thread-1-processing-n:127.0.0.1:56468_hz_%2Fw x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:56468_hz_%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:56468/hz_/w/collection1/ shard1
   [junit4]   2> 177227 INFO  (OverseerStateUpdate-96311457290387461-127.0.0.1:56468_hz_%2Fw-n_0000000000) [n:127.0.0.1:56468_hz_%2Fw    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"https://127.0.0.1:56468/hz_/w",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 1
   [junit4]   2> 177335 INFO  (zkCallback-247-thread-3-processing-n:127.0.0.1:56468_hz_%2Fw) [n:127.0.0.1:56468_hz_%2Fw    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 177379 INFO  (coreZkRegister-461-thread-1-processing-n:127.0.0.1:56468_hz_%2Fw x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:56468_hz_%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are https://127.0.0.1:56468/hz_/w/collection1/ and leader is https://127.0.0.1:56468/hz_/w/collection1/
   [junit4]   2> 177379 INFO  (coreZkRegister-461-thread-1-processing-n:127.0.0.1:56468_hz_%2Fw x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:56468_hz_%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:56468/hz_/w
   [junit4]   2> 177379 INFO  (coreZkRegister-461-thread-1-processing-n:127.0.0.1:56468_hz_%2Fw x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:56468_hz_%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 177379 INFO  (coreZkRegister-461-thread-1-processing-n:127.0.0.1:56468_hz_%2Fw x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:56468_hz_%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 177379 INFO  (coreZkRegister-461-thread-1-processing-n:127.0.0.1:56468_hz_%2Fw x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:56468_hz_%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 177380 INFO  (coreZkRegister-461-thread-1-processing-n:127.0.0.1:56468_hz_%2Fw x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:56468_hz_%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Checking legacy cluster state for collection control_collection
   [junit4]   2> 177381 INFO  (OverseerStateUpdate-96311457290387461-127.0.0.1:56468_hz_%2Fw-n_0000000000) [n:127.0.0.1:56468_hz_%2Fw    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:56468/hz_/w",
   [junit4]   2>   "node_name":"127.0.0.1:56468_hz_%2Fw",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"} current state version: 2
   [junit4]   2> 177381 INFO  (OverseerStateUpdate-96311457290387461-127.0.0.1:56468_hz_%2Fw-n_0000000000) [n:127.0.0.1:56468_hz_%2Fw    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:56468/hz_/w",
   [junit4]   2>   "node_name":"127.0.0.1:56468_hz_%2Fw",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 177467 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 177467 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 177473 INFO  (zkCallback-251-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4df33374 name:ZooKeeperConnection Watcher:127.0.0.1:46324/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 177473 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 177474 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 177474 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 177475 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [    ] o.a.s.c.c.ZkStateReader Loaded cluster properties: {urlScheme=https}
   [junit4]   2> 177475 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 177476 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 177476 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with stateFormat=2
   [junit4]   2> 177476 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 177484 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 177484 INFO  (zkCallback-247-thread-2-processing-n:127.0.0.1:56468_hz_%2Fw) [n:127.0.0.1:56468_hz_%2Fw    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 177484 INFO  (zkCallback-251-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 177485 INFO  (zkCallback-252-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@476924ed name:ZooKeeperConnection Watcher:127.0.0.1:46324/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 177485 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 177485 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 177488 INFO  (OverseerStateUpdate-96311457290387461-127.0.0.1:56468_hz_%2Fw-n_0000000000) [n:127.0.0.1:56468_hz_%2Fw    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"create",
   [junit4]   2>   "name":"collection1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "stateFormat":"2"} current state version: 3
   [junit4]   2> 177488 INFO  (OverseerStateUpdate-96311457290387461-127.0.0.1:56468_hz_%2Fw-n_0000000000) [n:127.0.0.1:56468_hz_%2Fw    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 177590 INFO  (OverseerStateUpdate-96311457290387461-127.0.0.1:56468_hz_%2Fw-n_0000000000) [n:127.0.0.1:56468_hz_%2Fw    ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/collection1/state.json
   [junit4]   2> 177611 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReqParamsAPI_10ABF5368D693666-001/shard-1-001/cores/collection1
   [junit4]   2> 177613 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReqParamsAPI_10ABF5368D693666-001/shard-1-001
   [junit4]   2> 177615 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [    ] o.e.j.s.Server jetty-9.3.8.v20160314
   [junit4]   2> 177629 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@165437d4{/hz_/w,null,AVAILABLE}
   [junit4]   2> 177631 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [    ] o.e.j.s.ServerConnector Started ServerConnector@1741de5d{SSL,[ssl, http/1.1]}{127.0.0.1:47216}
   [junit4]   2> 177631 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [    ] o.e.j.s.Server Started @180755ms
   [junit4]   2> 177631 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReqParamsAPI_10ABF5368D693666-001/tempDir-001/jetty1, solrconfig=solrconfig.xml, hostContext=/hz_/w, hostPort=47216, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReqParamsAPI_10ABF5368D693666-001/shard-1-001/cores}
   [junit4]   2> 177631 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@73d16e93
   [junit4]   2> 177631 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReqParamsAPI_10ABF5368D693666-001/shard-1-001'
   [junit4]   2> 177632 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 177632 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 177632 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 177633 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 177640 INFO  (zkCallback-253-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@60a65d2f name:ZooKeeperConnection Watcher:127.0.0.1:46324/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 177640 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 177640 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 177649 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 177650 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReqParamsAPI_10ABF5368D693666-001/shard-1-001/solr.xml
   [junit4]   2> 177656 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReqParamsAPI_10ABF5368D693666-001/shard-1-001/cores
   [junit4]   2> 177656 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [    ] o.a.s.c.CoreContainer New CoreContainer 1860857874
   [junit4]   2> 177656 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReqParamsAPI_10ABF5368D693666-001/shard-1-001]
   [junit4]   2> 177656 WARN  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReqParamsAPI_10ABF5368D693666-001/shard-1-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReqParamsAPI_10ABF5368D693666-001/shard-1-001/lib
   [junit4]   2> 177656 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,connectionsEvictorSleepDelay : 5000,maxConnectionIdleTime : 40000,
   [junit4]   2> 177666 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 177667 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 177668 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 177668 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46324/solr
   [junit4]   2> 177668 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 177668 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 177669 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 177678 INFO  (zkCallback-256-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3d13ee33 name:ZooKeeperConnection Watcher:127.0.0.1:46324 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 177678 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 177679 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 177680 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [n:127.0.0.1:47216_hz_%2Fw    ] o.a.s.c.ZkController Added new OnReconnect listener org.apache.solr.cloud.ZkController$$Lambda$50/377473286@57ad8a56
   [junit4]   2> 177681 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [n:127.0.0.1:47216_hz_%2Fw    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 177682 INFO  (zkCallback-257-thread-1-processing-n:127.0.0.1:47216_hz_%2Fw) [n:127.0.0.1:47216_hz_%2Fw    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1a45bffd name:ZooKeeperConnection Watcher:127.0.0.1:46324/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 177682 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [n:127.0.0.1:47216_hz_%2Fw    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 177686 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [n:127.0.0.1:47216_hz_%2Fw    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 177687 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [n:127.0.0.1:47216_hz_%2Fw    ] o.a.s.c.c.ZkStateReader Loaded cluster properties: {urlScheme=https}
   [junit4]   2> 177690 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [n:127.0.0.1:47216_hz_%2Fw    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 177692 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [n:127.0.0.1:47216_hz_%2Fw    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 177692 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [n:127.0.0.1:47216_hz_%2Fw    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/96311457290387466-127.0.0.1:47216_hz_%2Fw-n_0000000001
   [junit4]   2> 177693 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [n:127.0.0.1:47216_hz_%2Fw    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/96311457290387461-127.0.0.1:56468_hz_%2Fw-n_0000000000 to know if I could be the leader
   [junit4]   2> 177693 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [n:127.0.0.1:47216_hz_%2Fw    ] o.a.s.c.ZkController Publish node=127.0.0.1:47216_hz_%2Fw as DOWN
   [junit4]   2> 177694 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [n:127.0.0.1:47216_hz_%2Fw    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:47216_hz_%2Fw
   [junit4]   2> 177694 INFO  (TEST-TestReqParamsAPI.test-seed#[10ABF5368D693666]) [n:127.0.0.1:47216_hz_%2Fw    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:47216_hz_%2Fw
   [junit4]   2> 177695 INFO  (zkCallback-251-thread-1) [    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 177695 INFO  (zkCallback-251-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 177698 INFO  (zkCallback-247-thread-1-processing-n:127.0.0.1:56468_hz_%2Fw) [n:127.0.0.1:56468_hz_%2Fw    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 177698 INFO  (zkCallback-257-thread-1-processing-n:127.0.0.1:47216_hz_%2Fw) [n:127.0.0.1:47216_hz_%2Fw    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 177698 INFO  (zkCallback-247-thread-1-processing-n:127.0.0.1:56468_hz_%2Fw) [n:127.0.0.1:56468_hz_%2Fw    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 177698 INFO  (OverseerStateUpdate-96311457290387461-127.0.0.1:56468_hz_%2Fw-n_0000000000) [n:127.0.0.1:56468_hz_%2Fw    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"downnode",
   [junit4]   2>   "node_name":"127.0.0.1:47216_hz_%2Fw"} current state version: 3
   [junit4]   2> 177699 INFO  (OverseerStateUpdate-96311457290387461-127.0.0.1:56468_hz_%2Fw-n_0000000000) [n:127.0.0.1:56468_hz_%2Fw    ] o.a.s.c.o.NodeMutator DownNode state invoked for node: 127.0.0.1:47216_hz_%2Fw
   [junit4]   2> 177700 INFO  (zkCallback-251-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [2])
   [junit4]   2> 177700 INFO  (zkCallback-247-thread-1-processing-n:127.0.0.1:56468_hz_%2Fw) [n:127.0.0.1:56468_hz_%2Fw    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [2])
   [junit4]   2> 177703 INFO  (zkCallback-257-thread-1-processing-n:127.0.0.1:47216_hz_%2Fw) [n:127.0.0.1:47216_hz_%2Fw    ] o.a.s.c.c.ZkStateReader Updated live nodes from Zo

[...truncated too long message...]

0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-subtree-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-trigger-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-jdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-jdbm1-2.0.0-M2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-kerberos-codec-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-ldif-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-mavibot-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-protocol-kerberos-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-protocol-ldap-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-protocol-shared-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-xdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/api-all-1.0.0-M20.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/bcprov-jdk15-1.45.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/cglib-nodep-2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/commons-collections-3.2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/commons-math3-3.4.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/easymock-3.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/ehcache-core-2.4.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/hadoop-common-2.7.2-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/hadoop-hdfs-2.7.2-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/hadoop-minikdc-2.7.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/htrace-core-3.2.0-incubating.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jackson-annotations-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jackson-core-asl-1.9.13.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jackson-databind-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jackson-mapper-asl-1.9.13.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jersey-core-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jersey-server-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jetty-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jetty-util-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/mina-core-2.0.0-M5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/mockito-core-1.9.5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/netty-all-4.0.36.Final.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/objenesis-1.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/presto-parser-0.122.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/slice-0.10.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/analysis/icu/lucene-analyzers-icu-6.2.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/contrib/solr-analysis-extras/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/queryparser/classes/test:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/contrib/analysis-extras/lib/icu4j-56.1.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-launcher.jar:/x1/jenkins/.ant/lib/ivy-2.3.0.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-antlr.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-regexp.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-resolver.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-net.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jmf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-oro.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-xalan2.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-logging.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jsch.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit4.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-javamail.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jai.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jdepend.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bsf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-swing.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-netrexx.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bcel.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-testutil.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-log4j.jar:/x1/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.8/lib/tools.jar:/x1/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.3.4.jar com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/temp/junit4-J2-20160727_050711_309.events @/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/temp/junit4-J2-20160727_050711_309.suites -stdin
   [junit4] ERROR: JVM J2 ended with an exception: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1501)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:121)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:955)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:952)
   [junit4] 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4] 	at java.lang.Thread.run(Thread.java:745)

BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:771: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:707: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/common-build.xml:536: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/common-build.xml:1443: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/common-build.xml:984: At least one slave process threw an exception, first: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.

Total time: 483 minutes 8 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
ERROR: Failed to archive {solr/build/solr-core/test/temp/junit4-J1-20160727_050711_309.events=solr/build/solr-core/test/temp/junit4-J1-20160727_050711_309.events, README.txt=README.txt, solr/build/solr-core/test/temp/junit4-J0-20160727_050711_309.events=solr/build/solr-core/test/temp/junit4-J0-20160727_050711_309.events, heapdumps/java_pid6266.hprof=heapdumps/java_pid6266.hprof, heapdumps/java_pid6267.hprof=heapdumps/java_pid6267.hprof, solr/build/solr-core/test/temp/junit4-J2-20160727_050711_309.events=solr/build/solr-core/test/temp/junit4-J2-20160727_050711_309.events} due to internal error; falling back to full archiving
java.lang.NullPointerException
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any




[JENKINS] Lucene-Solr-NightlyTests-6.x - Build # 126 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-6.x/126/

No tests ran.

Build Log:
[...truncated 11284 lines...]
   [junit4] Suite: org.apache.solr.cloud.CollectionStateFormat2Test
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionStateFormat2Test_21913F6461EA0D1C-001/init-core-data-001
   [junit4]   2> 271973 INFO  (SUITE-CollectionStateFormat2Test-seed#[21913F6461EA0D1C]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
   [junit4]   2> 271974 INFO  (SUITE-CollectionStateFormat2Test-seed#[21913F6461EA0D1C]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   2> 271995 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 271995 INFO  (Thread-299) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 271995 INFO  (Thread-299) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 272095 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [    ] o.a.s.c.ZkTestServer start zk server on port:56737
   [junit4]   2> 272095 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 272110 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 272130 INFO  (zkCallback-274-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@415fce38 name:ZooKeeperConnection Watcher:127.0.0.1:56737 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 272131 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 272131 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 272131 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 272135 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 272141 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 272159 INFO  (zkCallback-275-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6e6b41a8 name:ZooKeeperConnection Watcher:127.0.0.1:56737/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 272159 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 272160 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 272160 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 272161 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 272162 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 272163 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 272164 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 272164 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 272166 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 272166 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 272167 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 272168 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 272169 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 272169 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 272170 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 272170 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 272171 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 272171 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 272172 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 272173 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 272181 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 272181 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 272200 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 272200 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 272202 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 272202 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 272203 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 272203 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 272211 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 272220 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 272254 INFO  (zkCallback-277-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@c30b721 name:ZooKeeperConnection Watcher:127.0.0.1:56737/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 272254 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 272254 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 272461 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionStateFormat2Test_21913F6461EA0D1C-001/control-001/cores/collection1
   [junit4]   2> 272464 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [    ] o.e.j.s.Server jetty-9.3.8.v20160314
   [junit4]   2> 272466 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@25bf04f1{/,null,AVAILABLE}
   [junit4]   2> 272473 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [    ] o.e.j.s.ServerConnector Started ServerConnector@18fcc60{SSL,[ssl, http/1.1]}{127.0.0.1:37788}
   [junit4]   2> 272473 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [    ] o.e.j.s.Server Started @277015ms
   [junit4]   2> 272473 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionStateFormat2Test_21913F6461EA0D1C-001/tempDir-001/control/data, hostContext=/, hostPort=37788, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionStateFormat2Test_21913F6461EA0D1C-001/control-001/cores}
   [junit4]   2> 272473 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@73d16e93
   [junit4]   2> 272473 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionStateFormat2Test_21913F6461EA0D1C-001/control-001'
   [junit4]   2> 272473 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 272473 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 272474 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 272474 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 272475 INFO  (zkCallback-278-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3befc1e3 name:ZooKeeperConnection Watcher:127.0.0.1:56737/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 272475 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 272475 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 272476 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 272476 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionStateFormat2Test_21913F6461EA0D1C-001/control-001/solr.xml
   [junit4]   2> 272482 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionStateFormat2Test_21913F6461EA0D1C-001/control-001/cores
   [junit4]   2> 272482 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [    ] o.a.s.c.CoreContainer New CoreContainer 405823038
   [junit4]   2> 272482 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionStateFormat2Test_21913F6461EA0D1C-001/control-001]
   [junit4]   2> 272482 WARN  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionStateFormat2Test_21913F6461EA0D1C-001/control-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionStateFormat2Test_21913F6461EA0D1C-001/control-001/lib
   [junit4]   2> 272482 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,connectionsEvictorSleepDelay : 5000,maxConnectionIdleTime : 40000,
   [junit4]   2> 272487 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 272488 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 272488 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 272488 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:56737/solr
   [junit4]   2> 272488 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 272489 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 272492 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 272493 INFO  (zkCallback-281-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@685f50e name:ZooKeeperConnection Watcher:127.0.0.1:56737 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 272493 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 272493 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 272497 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [n:127.0.0.1:37788_    ] o.a.s.c.ZkController Added new OnReconnect listener org.apache.solr.cloud.ZkController$$Lambda$83/1159850662@6b61537a
   [junit4]   2> 272498 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [n:127.0.0.1:37788_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 272499 INFO  (zkCallback-282-thread-1-processing-n:127.0.0.1:37788_) [n:127.0.0.1:37788_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7fb17a7c name:ZooKeeperConnection Watcher:127.0.0.1:56737/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 272499 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [n:127.0.0.1:37788_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 272500 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [n:127.0.0.1:37788_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 272501 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [n:127.0.0.1:37788_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 272504 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [n:127.0.0.1:37788_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 272505 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [n:127.0.0.1:37788_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 272507 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [n:127.0.0.1:37788_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 272508 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [n:127.0.0.1:37788_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 272510 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [n:127.0.0.1:37788_    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 272511 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [n:127.0.0.1:37788_    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 272512 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [n:127.0.0.1:37788_    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 272513 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [n:127.0.0.1:37788_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 272513 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [n:127.0.0.1:37788_    ] o.a.s.c.c.ZkStateReader Loaded cluster properties: {urlScheme=https}
   [junit4]   2> 272513 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [n:127.0.0.1:37788_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (0)
   [junit4]   2> 272516 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [n:127.0.0.1:37788_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 272517 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [n:127.0.0.1:37788_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 272518 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [n:127.0.0.1:37788_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 272518 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [n:127.0.0.1:37788_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/96287201236353029-127.0.0.1:37788_-n_0000000000
   [junit4]   2> 272519 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [n:127.0.0.1:37788_    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:37788_
   [junit4]   2> 272519 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [n:127.0.0.1:37788_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 272520 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [n:127.0.0.1:37788_    ] o.a.s.c.Overseer Overseer (id=96287201236353029-127.0.0.1:37788_-n_0000000000) starting
   [junit4]   2> 272521 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [n:127.0.0.1:37788_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 272529 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [n:127.0.0.1:37788_    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 272530 INFO  (OverseerCollectionConfigSetProcessor-96287201236353029-127.0.0.1:37788_-n_0000000000) [n:127.0.0.1:37788_    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 272530 INFO  (OverseerStateUpdate-96287201236353029-127.0.0.1:37788_-n_0000000000) [n:127.0.0.1:37788_    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 272531 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [n:127.0.0.1:37788_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:37788_
   [junit4]   2> 272531 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [n:127.0.0.1:37788_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:37788_
   [junit4]   2> 272532 INFO  (OverseerStateUpdate-96287201236353029-127.0.0.1:37788_-n_0000000000) [n:127.0.0.1:37788_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (0)
   [junit4]   2> 272534 INFO  (zkCallback-282-thread-1-processing-n:127.0.0.1:37788_) [n:127.0.0.1:37788_    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [0])
   [junit4]   2> 272534 INFO  (zkCallback-282-thread-1-processing-n:127.0.0.1:37788_) [n:127.0.0.1:37788_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 272541 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [n:127.0.0.1:37788_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 272541 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [n:127.0.0.1:37788_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 272542 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [n:127.0.0.1:37788_    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionStateFormat2Test_21913F6461EA0D1C-001/control-001/cores
   [junit4]   2> 272543 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [n:127.0.0.1:37788_    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, loadOnStartup=true, configSetProperties=configsetprops.json, collection=control_collection, coreNodeName=, dataDir=data/, shard=}
   [junit4]   2> 272543 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [n:127.0.0.1:37788_    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionStateFormat2Test_21913F6461EA0D1C-001/control-001/cores/collection1
   [junit4]   2> 272543 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [n:127.0.0.1:37788_    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 272544 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [n:127.0.0.1:37788_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2
   [junit4]   2> 272544 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [n:127.0.0.1:37788_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 272545 INFO  (coreLoadExecutor-642-thread-1-processing-n:127.0.0.1:37788_) [n:127.0.0.1:37788_ c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 272545 INFO  (coreLoadExecutor-642-thread-1-processing-n:127.0.0.1:37788_) [n:127.0.0.1:37788_ c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 272546 INFO  (coreLoadExecutor-642-thread-1-processing-n:127.0.0.1:37788_) [n:127.0.0.1:37788_ c:control_collection   x:collection1] o.a.s.c.ZkController Collection control_collection not visible yet, but flagging it so a watch is registered when it becomes visible
   [junit4]   2> 272546 INFO  (OverseerStateUpdate-96287201236353029-127.0.0.1:37788_-n_0000000000) [n:127.0.0.1:37788_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:37788",
   [junit4]   2>   "node_name":"127.0.0.1:37788_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 272547 INFO  (OverseerStateUpdate-96287201236353029-127.0.0.1:37788_-n_0000000000) [n:127.0.0.1:37788_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:37788",
   [junit4]   2>   "node_name":"127.0.0.1:37788_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 272547 INFO  (OverseerStateUpdate-96287201236353029-127.0.0.1:37788_-n_0000000000) [n:127.0.0.1:37788_    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 272547 INFO  (coreLoadExecutor-642-thread-1-processing-n:127.0.0.1:37788_) [n:127.0.0.1:37788_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Deleting data for [control_collection]
   [junit4]   2> 272548 INFO  (OverseerStateUpdate-96287201236353029-127.0.0.1:37788_-n_0000000000) [n:127.0.0.1:37788_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 272549 INFO  (zkCallback-282-thread-1-processing-n:127.0.0.1:37788_) [n:127.0.0.1:37788_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 272549 INFO  (coreLoadExecutor-642-thread-1-processing-n:127.0.0.1:37788_) [n:127.0.0.1:37788_ c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 273550 INFO  (coreLoadExecutor-642-thread-1-processing-n:127.0.0.1:37788_) [n:127.0.0.1:37788_ c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 273550 INFO  (coreLoadExecutor-642-thread-1-processing-n:127.0.0.1:37788_) [n:127.0.0.1:37788_ c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 273550 INFO  (coreLoadExecutor-642-thread-1-processing-n:127.0.0.1:37788_) [n:127.0.0.1:37788_ c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 273550 INFO  (coreLoadExecutor-642-thread-1-processing-n:127.0.0.1:37788_) [n:127.0.0.1:37788_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/control_collection]
   [junit4]   2> 273551 INFO  (coreLoadExecutor-642-thread-1-processing-n:127.0.0.1:37788_) [n:127.0.0.1:37788_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/control_collection] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 273551 INFO  (coreLoadExecutor-642-thread-1-processing-n:127.0.0.1:37788_) [n:127.0.0.1:37788_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionStateFormat2Test_21913F6461EA0D1C-001/control-001/cores/collection1'
   [junit4]   2> 273551 INFO  (coreLoadExecutor-642-thread-1-processing-n:127.0.0.1:37788_) [n:127.0.0.1:37788_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 273551 INFO  (coreLoadExecutor-642-thread-1-processing-n:127.0.0.1:37788_) [n:127.0.0.1:37788_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 273553 INFO  (coreLoadExecutor-642-thread-1-processing-n:127.0.0.1:37788_) [n:127.0.0.1:37788_ c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 273560 INFO  (coreLoadExecutor-642-thread-1-processing-n:127.0.0.1:37788_) [n:127.0.0.1:37788_ c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 273564 INFO  (coreLoadExecutor-642-thread-1-processing-n:127.0.0.1:37788_) [n:127.0.0.1:37788_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.2.0
   [junit4]   2> 273574 INFO  (coreLoadExecutor-642-thread-1-processing-n:127.0.0.1:37788_) [n:127.0.0.1:37788_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 273579 INFO  (coreLoadExecutor-642-thread-1-processing-n:127.0.0.1:37788_) [n:127.0.0.1:37788_ c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 273665 INFO  (coreLoadExecutor-642-thread-1-processing-n:127.0.0.1:37788_) [n:127.0.0.1:37788_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 273671 INFO  (coreLoadExecutor-642-thread-1-processing-n:127.0.0.1:37788_) [n:127.0.0.1:37788_ c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 273672 INFO  (coreLoadExecutor-642-thread-1-processing-n:127.0.0.1:37788_) [n:127.0.0.1:37788_ c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 273681 INFO  (coreLoadExecutor-642-thread-1-processing-n:127.0.0.1:37788_) [n:127.0.0.1:37788_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 273683 INFO  (coreLoadExecutor-642-thread-1-processing-n:127.0.0.1:37788_) [n:127.0.0.1:37788_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 273684 INFO  (coreLoadExecutor-642-thread-1-processing-n:127.0.0.1:37788_) [n:127.0.0.1:37788_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 273685 INFO  (coreLoadExecutor-642-thread-1-processing-n:127.0.0.1:37788_) [n:127.0.0.1:37788_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 273686 INFO  (coreLoadExecutor-642-thread-1-processing-n:127.0.0.1:37788_) [n:127.0.0.1:37788_ c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2
   [junit4]   2> 273686 INFO  (coreLoadExecutor-642-thread-1-processing-n:127.0.0.1:37788_) [n:127.0.0.1:37788_ c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 273687 INFO  (coreLoadExecutor-642-thread-1-processing-n:127.0.0.1:37788_) [n:127.0.0.1:37788_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 273687 INFO  (coreLoadExecutor-642-thread-1-processing-n:127.0.0.1:37788_) [n:127.0.0.1:37788_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionStateFormat2Test_21913F6461EA0D1C-001/control-001/cores/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionStateFormat2Test_21913F6461EA0D1C-001/control-001/cores/collection1/data/]
   [junit4]   2> 273687 INFO  (coreLoadExecutor-642-thread-1-processing-n:127.0.0.1:37788_) [n:127.0.0.1:37788_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@5484988
   [junit4]   2> 273687 INFO  (coreLoadExecutor-642-thread-1-processing-n:127.0.0.1:37788_) [n:127.0.0.1:37788_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionStateFormat2Test_21913F6461EA0D1C-001/control-001/cores/collection1/data
   [junit4]   2> 273687 INFO  (coreLoadExecutor-642-thread-1-processing-n:127.0.0.1:37788_) [n:127.0.0.1:37788_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionStateFormat2Test_21913F6461EA0D1C-001/control-001/cores/collection1/data/index/
   [junit4]   2> 273688 WARN  (coreLoadExecutor-642-thread-1-processing-n:127.0.0.1:37788_) [n:127.0.0.1:37788_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionStateFormat2Test_21913F6461EA0D1C-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 273688 INFO  (coreLoadExecutor-642-thread-1-processing-n:127.0.0.1:37788_) [n:127.0.0.1:37788_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionStateFormat2Test_21913F6461EA0D1C-001/control-001/cores/collection1/data/index
   [junit4]   2> 273688 INFO  (coreLoadExecutor-642-thread-1-processing-n:127.0.0.1:37788_) [n:127.0.0.1:37788_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=38, maxMergeAtOnceExplicit=39, maxMergedSegmentMB=60.9931640625, floorSegmentMB=1.7939453125, forceMergeDeletesPctAllowed=17.42902087228664, segmentsPerTier=34.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 273689 INFO  (coreLoadExecutor-642-thread-1-processing-n:127.0.0.1:37788_) [n:127.0.0.1:37788_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@63c462ab lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5092d6b9),segFN=segments_1,generation=1}
   [junit4]   2> 273689 INFO  (coreLoadExecutor-642-thread-1-processing-n:127.0.0.1:37788_) [n:127.0.0.1:37788_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 273689 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:37788_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionStateFormat2Test_21913F6461EA0D1C-001/control-001/cores/collection1/data/
   [junit4]   2> 273690 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:37788_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionStateFormat2Test_21913F6461EA0D1C-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 273692 INFO  (coreLoadExecutor-642-thread-1-processing-n:127.0.0.1:37788_) [n:127.0.0.1:37788_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 273693 INFO  (coreLoadExecutor-642-thread-1-processing-n:127.0.0.1:37788_) [n:127.0.0.1:37788_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 273693 INFO  (coreLoadExecutor-642-thread-1-processing-n:127.0.0.1:37788_) [n:127.0.0.1:37788_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 273693 INFO  (coreLoadExecutor-642-thread-1-processing-n:127.0.0.1:37788_) [n:127.0.0.1:37788_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 273693 INFO  (coreLoadExecutor-642-thread-1-processing-n:127.0.0.1:37788_) [n:127.0.0.1:37788_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 273693 INFO  (coreLoadExecutor-642-thread-1-processing-n:127.0.0.1:37788_) [n:127.0.0.1:37788_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 273693 INFO  (coreLoadExecutor-642-thread-1-processing-n:127.0.0.1:37788_) [n:127.0.0.1:37788_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 273693 INFO  (coreLoadExecutor-642-thread-1-processing-n:127.0.0.1:37788_) [n:127.0.0.1:37788_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 273693 INFO  (coreLoadExecutor-642-thread-1-processing-n:127.0.0.1:37788_) [n:127.0.0.1:37788_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 273693 WARN  (coreLoadExecutor-642-thread-1-processing-n:127.0.0.1:37788_) [n:127.0.0.1:37788_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
   [junit4]   2> 273695 INFO  (coreLoadExecutor-642-thread-1-processing-n:127.0.0.1:37788_) [n:127.0.0.1:37788_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 273696 INFO  (coreLoadExecutor-642-thread-1-processing-n:127.0.0.1:37788_) [n:127.0.0.1:37788_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 273696 INFO  (coreLoadExecutor-642-thread-1-processing-n:127.0.0.1:37788_) [n:127.0.0.1:37788_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 273697 INFO  (coreLoadExecutor-642-thread-1-processing-n:127.0.0.1:37788_) [n:127.0.0.1:37788_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 273701 INFO  (coreLoadExecutor-642-thread-1-processing-n:127.0.0.1:37788_) [n:127.0.0.1:37788_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/mbeans,standard,/update/json/docs,/admin/luke,/export,/get,/admin/properties,/update/json,/admin/threads,/dump,/update/csv,/sql,/graph,/admin/segments,/admin/system,/replication,/config,/stream,/schema,/admin/plugins,/admin/logging,/admin/ping,/update,/admin/file,/terms
   [junit4]   2> 273701 INFO  (coreLoadExecutor-642-thread-1-processing-n:127.0.0.1:37788_) [n:127.0.0.1:37788_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 273701 INFO  (coreLoadExecutor-642-thread-1-processing-n:127.0.0.1:37788_) [n:127.0.0.1:37788_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 273701 INFO  (coreLoadExecutor-642-thread-1-processing-n:127.0.0.1:37788_) [n:127.0.0.1:37788_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 273702 INFO  (coreLoadExecutor-642-thread-1-processing-n:127.0.0.1:37788_) [n:127.0.0.1:37788_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 273702 INFO  (coreLoadExecutor-642-thread-1-processing-n:127.0.0.1:37788_) [n:127.0.0.1:37788_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 273703 INFO  (coreLoadExecutor-642-thread-1-processing-n:127.0.0.1:37788_) [n:127.0.0.1:37788_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=47, maxMergeAtOnceExplicit=38, maxMergedSegmentMB=37.77734375, floorSegmentMB=1.322265625, forceMergeDeletesPctAllowed=6.813569310661979, segmentsPerTier=33.0, maxCFSSegmentSizeMB=1.201171875, noCFSRatio=0.0
   [junit4]   2> 273703 INFO  (coreLoadExecutor-642-thread-1-processing-n:127.0.0.1:37788_) [n:127.0.0.1:37788_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@63c462ab lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5092d6b9),segFN=segments_1,generation=1}
   [junit4]   2> 273703 INFO  (coreLoadExecutor-642-thread-1-processing-n:127.0.0.1:37788_) [n:127.0.0.1:37788_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 273703 INFO  (coreLoadExecutor-642-thread-1-processing-n:127.0.0.1:37788_) [n:127.0.0.1:37788_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@6aceab0e[collection1] main]
   [junit4]   2> 273703 INFO  (coreLoadExecutor-642-thread-1-processing-n:127.0.0.1:37788_) [n:127.0.0.1:37788_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/control_collection]
   [junit4]   2> 273706 INFO  (coreLoadExecutor-642-thread-1-processing-n:127.0.0.1:37788_) [n:127.0.0.1:37788_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/control_collection] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 273706 INFO  (coreLoadExecutor-642-thread-1-processing-n:127.0.0.1:37788_) [n:127.0.0.1:37788_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 273706 INFO  (coreLoadExecutor-642-thread-1-processing-n:127.0.0.1:37788_) [n:127.0.0.1:37788_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 273706 INFO  (coreLoadExecutor-642-thread-1-processing-n:127.0.0.1:37788_) [n:127.0.0.1:37788_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 273706 INFO  (coreLoadExecutor-642-thread-1-processing-n:127.0.0.1:37788_) [n:127.0.0.1:37788_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 273706 INFO  (coreLoadExecutor-642-thread-1-processing-n:127.0.0.1:37788_) [n:127.0.0.1:37788_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 273706 INFO  (coreLoadExecutor-642-thread-1-processing-n:127.0.0.1:37788_) [n:127.0.0.1:37788_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 273707 INFO  (coreLoadExecutor-642-thread-1-processing-n:127.0.0.1:37788_) [n:127.0.0.1:37788_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 273707 INFO  (coreLoadExecutor-642-thread-1-processing-n:127.0.0.1:37788_) [n:127.0.0.1:37788_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 273707 INFO  (searcherExecutor-643-thread-1-processing-n:127.0.0.1:37788_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:37788_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@6aceab0e[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 273707 INFO  (coreLoadExecutor-642-thread-1-processing-n:127.0.0.1:37788_) [n:127.0.0.1:37788_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 273707 INFO  (coreLoadExecutor-642-thread-1-processing-n:127.0.0.1:37788_) [n:127.0.0.1:37788_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 273707 INFO  (coreLoadExecutor-642-thread-1-processing-n:127.0.0.1:37788_) [n:127.0.0.1:37788_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 273707 INFO  (coreLoadExecutor-642-thread-1-processing-n:127.0.0.1:37788_) [n:127.0.0.1:37788_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1540595221575761920
   [junit4]   2> 273710 INFO  (coreLoadExecutor-642-thread-1-processing-n:127.0.0.1:37788_) [n:127.0.0.1:37788_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 2.0ms to seed version buckets with highest version 1540595221575761920
   [junit4]   2> 273710 INFO  (coreLoadExecutor-642-thread-1-processing-n:127.0.0.1:37788_) [n:127.0.0.1:37788_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 273711 INFO  (coreLoadExecutor-642-thread-1-processing-n:127.0.0.1:37788_) [n:127.0.0.1:37788_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 273712 INFO  (coreZkRegister-636-thread-1-processing-n:127.0.0.1:37788_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:37788_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:https://127.0.0.1:37788 collection:control_collection shard:shard1
   [junit4]   2> 273713 INFO  (coreZkRegister-636-thread-1-processing-n:127.0.0.1:37788_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:37788_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 273716 INFO  (coreZkRegister-636-thread-1-processing-n:127.0.0.1:37788_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:37788_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/96287201236353029-core_node1-n_0000000000
   [junit4]   2> 273716 INFO  (coreZkRegister-636-thread-1-processing-n:127.0.0.1:37788_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:37788_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard=shard1 and weAreReplacement=false and leaderVoteWait=10000
   [junit4]   2> 273718 INFO  (coreZkRegister-636-thread-1-processing-n:127.0.0.1:37788_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:37788_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 273718 INFO  (coreZkRegister-636-thread-1-processing-n:127.0.0.1:37788_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:37788_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 273718 INFO  (coreZkRegister-636-thread-1-processing-n:127.0.0.1:37788_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:37788_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:37788/collection1/
   [junit4]   2> 273718 INFO  (coreZkRegister-636-thread-1-processing-n:127.0.0.1:37788_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:37788_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 273718 INFO  (OverseerStateUpdate-96287201236353029-127.0.0.1:37788_-n_0000000000) [n:127.0.0.1:37788_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 273718 INFO  (coreZkRegister-636-thread-1-processing-n:127.0.0.1:37788_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:37788_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy https://127.0.0.1:37788/collection1/ has no replicas
   [junit4]   2> 273719 INFO  (coreZkRegister-636-thread-1-processing-n:127.0.0.1:37788_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:37788_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 273721 INFO  (coreZkRegister-636-thread-1-processing-n:127.0.0.1:37788_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:37788_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/control_collection/leaders/shard1/leader after winning as /collections/control_collection/leader_elect/shard1/election/96287201236353029-core_node1-n_0000000000
   [junit4]   2> 273724 INFO  (coreZkRegister-636-thread-1-processing-n:127.0.0.1:37788_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:37788_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:37788/collection1/ shard1
   [junit4]   2> 273725 INFO  (OverseerStateUpdate-96287201236353029-127.0.0.1:37788_-n_0000000000) [n:127.0.0.1:37788_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"https://127.0.0.1:37788",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 1
   [junit4]   2> 273827 INFO  (zkCallback-282-thread-3-processing-n:127.0.0.1:37788_) [n:127.0.0.1:37788_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 273875 INFO  (coreZkRegister-636-thread-1-processing-n:127.0.0.1:37788_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:37788_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are https://127.0.0.1:37788/collection1/ and leader is https://127.0.0.1:37788/collection1/
   [junit4]   2> 273875 INFO  (coreZkRegister-636-thread-1-processing-n:127.0.0.1:37788_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:37788_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:37788
   [junit4]   2> 273875 INFO  (coreZkRegister-636-thread-1-processing-n:127.0.0.1:37788_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:37788_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 273875 INFO  (coreZkRegister-636-thread-1-processing-n:127.0.0.1:37788_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:37788_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 273875 INFO  (coreZkRegister-636-thread-1-processing-n:127.0.0.1:37788_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:37788_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 273876 INFO  (coreZkRegister-636-thread-1-processing-n:127.0.0.1:37788_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:37788_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Checking legacy cluster state for collection control_collection
   [junit4]   2> 273877 INFO  (OverseerStateUpdate-96287201236353029-127.0.0.1:37788_-n_0000000000) [n:127.0.0.1:37788_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:37788",
   [junit4]   2>   "node_name":"127.0.0.1:37788_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"} current state version: 2
   [junit4]   2> 273877 INFO  (OverseerStateUpdate-96287201236353029-127.0.0.1:37788_-n_0000000000) [n:127.0.0.1:37788_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:37788",
   [junit4]   2>   "node_name":"127.0.0.1:37788_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 273979 INFO  (zkCallback-282-thread-3-processing-n:127.0.0.1:37788_) [n:127.0.0.1:37788_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 274046 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 274046 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 274048 INFO  (zkCallback-286-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3bb3519c name:ZooKeeperConnection Watcher:127.0.0.1:56737/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 274048 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 274048 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 274048 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 274049 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [    ] o.a.s.c.c.ZkStateReader Loaded cluster properties: {urlScheme=https}
   [junit4]   2> 274049 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 274050 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 274050 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with stateFormat=2
   [junit4]   2> 274050 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 274051 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 274051 INFO  (zkCallback-287-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@14caba9c name:ZooKeeperConnection Watcher:127.0.0.1:56737/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 274051 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 274052 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 274054 INFO  (OverseerStateUpdate-96287201236353029-127.0.0.1:37788_-n_0000000000) [n:127.0.0.1:37788_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"create",
   [junit4]   2>   "name":"collection1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "stateFormat":"2"} current state version: 3
   [junit4]   2> 274054 INFO  (OverseerStateUpdate-96287201236353029-127.0.0.1:37788_-n_0000000000) [n:127.0.0.1:37788_    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 274144 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionStateFormat2Test_21913F6461EA0D1C-001/shard-1-001/cores/collection1
   [junit4]   2> 274145 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionStateFormat2Test_21913F6461EA0D1C-001/shard-1-001
   [junit4]   2> 274147 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [    ] o.e.j.s.Server jetty-9.3.8.v20160314
   [junit4]   2> 274149 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5f9f187e{/,null,AVAILABLE}
   [junit4]   2> 274150 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [    ] o.e.j.s.ServerConnector Started ServerConnector@53133dd3{SSL,[ssl, http/1.1]}{127.0.0.1:37236}
   [junit4]   2> 274151 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [    ] o.e.j.s.Server Started @278693ms
   [junit4]   2> 274151 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionStateFormat2Test_21913F6461EA0D1C-001/tempDir-001/jetty1, solrconfig=solrconfig.xml, hostContext=/, hostPort=37236, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionStateFormat2Test_21913F6461EA0D1C-001/shard-1-001/cores}
   [junit4]   2> 274151 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@73d16e93
   [junit4]   2> 274151 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionStateFormat2Test_21913F6461EA0D1C-001/shard-1-001'
   [junit4]   2> 274151 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 274151 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 274151 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 274152 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 274153 INFO  (zkCallback-288-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@789c881c name:ZooKeeperConnection Watcher:127.0.0.1:56737/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 274154 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 274154 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 274156 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 274156 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionStateFormat2Test_21913F6461EA0D1C-001/shard-1-001/solr.xml
   [junit4]   2> 274160 INFO  (OverseerStateUpdate-96287201236353029-127.0.0.1:37788_-n_0000000000) [n:127.0.0.1:37788_    ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/collection1/state.json
   [junit4]   2> 274161 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionStateFormat2Test_21913F6461EA0D1C-001/shard-1-001/cores
   [junit4]   2> 274161 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [    ] o.a.s.c.CoreContainer New CoreContainer 35437886
   [junit4]   2> 274161 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionStateFormat2Test_21913F6461EA0D1C-001/shard-1-001]
   [junit4]   2> 274161 WARN  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionStateFormat2Test_21913F6461EA0D1C-001/shard-1-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionStateFormat2Test_21913F6461EA0D1C-001/shard-1-001/lib
   [junit4]   2> 274161 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,connectionsEvictorSleepDelay : 5000,maxConnectionIdleTime : 40000,
   [junit4]   2> 274163 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 274163 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 274163 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 274163 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:56737/solr
   [junit4]   2> 274163 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 274163 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 274164 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 274165 INFO  (zkCallback-291-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@52657e9d name:ZooKeeperConnection Watcher:127.0.0.1:56737 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 274165 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 274166 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 274166 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [n:127.0.0.1:37236_    ] o.a.s.c.ZkController Added new OnReconnect listener org.apache.solr.cloud.ZkController$$Lambda$83/1159850662@2521911b
   [junit4]   2> 274167 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [n:127.0.0.1:37236_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 274168 INFO  (zkCallback-292-thread-1-processing-n:127.0.0.1:37236_) [n:127.0.0.1:37236_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4a0e74cc name:ZooKeeperConnection Watcher:127.0.0.1:56737/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 274168 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [n:127.0.0.1:37236_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 274172 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [n:127.0.0.1:37236_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 274173 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [n:127.0.0.1:37236_    ] o.a.s.c.c.ZkStateReader Loaded cluster properties: {urlScheme=https}
   [junit4]   2> 274173 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [n:127.0.0.1:37236_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 274176 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [n:127.0.0.1:37236_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 274176 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [n:127.0.0.1:37236_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/96287201236353034-127.0.0.1:37236_-n_0000000001
   [junit4]   2> 274176 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [n:127.0.0.1:37236_    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/96287201236353029-127.0.0.1:37788_-n_0000000000 to know if I could be the leader
   [junit4]   2> 274177 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [n:127.0.0.1:37236_    ] o.a.s.c.ZkController Publish node=127.0.0.1:37236_ as DOWN
   [junit4]   2> 274178 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [n:127.0.0.1:37236_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:37236_
   [junit4]   2> 274178 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [n:127.0.0.1:37236_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:37236_
   [junit4]   2> 274178 INFO  (OverseerStateUpdate-96287201236353029-127.0.0.1:37788_-n_0000000000) [n:127.0.0.1:37788_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"downnode",
   [junit4]   2>   "node_name":"127.0.0.1:37236_"} current state version: 3
   [junit4]   2> 274178 INFO  (OverseerStateUpdate-96287201236353029-127.0.0.1:37788_-n_0000000000) [n:127.0.0.1:37788_    ] o.a.s.c.o.NodeMutator DownNode state invoked for node: 127.0.0.1:37236_
   [junit4]   2> 274179 INFO  (zkCallback-286-thread-1) [    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 274179 INFO  (zkCallback-282-thread-3-processing-n:127.0.0.1:37788_) [n:127.0.0.1:37788_    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 274179 INFO  (zkCallback-286-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 274180 INFO  (zkCallback-292-thread-1-processing-n:127.0.0.1:37236_) [n:127.0.0.1:37236_    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 274180 INFO  (zkCallback-282-thread-3-processing-n:127.0.0.1:37788_) [n:127.0.0.1:37788_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 274180 INFO  (zkCallback-282-thread-3-processing-n:127.0.0.1:37788_) [n:127.0.0.1:37788_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [2])
   [junit4]   2> 274181 INFO  (zkCallback-286-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [2])
   [junit4]   2> 274185 INFO  (zkCallback-292-thread-2-processing-n:127.0.0.1:37236_) [n:127.0.0.1:37236_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 274189 INFO  (zkCallback-292-thread-1-processing-n:127.0.0.1:37236_) [n:127.0.0.1:37236_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 274192 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [n:127.0.0.1:37236_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 274192 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [n:127.0.0.1:37236_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 274193 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [n:127.0.0.1:37236_    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionStateFormat2Test_21913F6461EA0D1C-001/shard-1-001/cores
   [junit4]   2> 274193 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [n:127.0.0.1:37236_    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, loadOnStartup=true, configSetProperties=configsetprops.json, collection=collection1, coreNodeName=, dataDir=data/, shard=}
   [junit4]   2> 274194 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [n:127.0.0.1:37236_    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionStateFormat2Test_21913F6461EA0D1C-001/shard-1-001/cores/collection1
   [junit4]   2> 274194 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [n:127.0.0.1:37236_    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 274195 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [n:127.0.0.1:37236_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2
   [junit4]   2> 274195 INFO  (TEST-CollectionStateFormat2Test.test-seed#[21913F6461EA0D1C]) [n:127.0.0.1:37236_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 274196 INFO  (coreLoadExecutor-653-thread-1-processing-n:127.0.0.1:37236_) [n:127.0.0.1:37236_ c:collection1   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 274196 INFO  (coreLoadExecutor-653-thread-1-processing-n:127.0.0.1:37236_) [n:127.0.0.1:37236_ c:collection1   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 274197 INFO  (coreLoadExecutor-653-thread-1-processing-n:127.0.0.1:37236_) [n:127.0.0.1:37236_ c:collection1   x:collection1] o.a.s.c.ZkController Registering watch for collection collection1
   [junit4]   2> 274197 INFO  (OverseerStateUpdate-96287201236353029-127.0.0.1:37788_-n_0000000000) [n:127.0.0.1:37788_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:37236",
   [junit4]   2>   "node_name":"127.0.0.1:37236_",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "operation":"state"} current state version: 4
   [junit4]   2> 274197 INFO  (OverseerStateUpdate-96287201236353029-127.0.0.1:37788_-n_0000000000) [n:127.0.0.1:37788_    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:37236",
   [junit4]   2>   "node_name":"127.0.0.1:37236_",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 274197 INFO  (OverseerStateUpdate-96287201236353029-127.0.0.1:37788_-n_0000000000) [n:127.0.0.1:37788_    ] o.a.s.c.o.ReplicaMutator Collection already exists with numShards=2
   [junit4]   2> 274197 INFO  (OverseerStateUpdate-96287201236353029-127.0.0.1:37788_-n_0000000000) [n:127.0.0.1:37788_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2
   [junit4]   2> 274197 INFO  (coreLoadExecutor-653-thread-1-processing-n:127.0.0.1:37236_) [n:127.0.0.1:37236_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Add data for [collection1] ver [0]
   [junit4]   2> 274198 INFO  (coreLoadExecutor-653-thread-1-processing-n:127.0.0.1:37236_) [n:127.0.0.1:37236_ c:collection1   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 274299 INFO  (OverseerStateUpdate-96287201236353029-127.0.0.1:37788_-n_0000000000) [n:127.0.0.1:37788_    ] o.a.s.c.o.ZkStateWriter going to update_collection /collections/collection1/state.json version: 0
   [junit4]   2> 274300 INFO  (zkCallback-292-thread-1-processing-n:127.0.0.1:37236_) [n:127.0.0.1:37236_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 274300 INFO  (zkCallback-292-thread-1-processing-n:127.0.0.1:37236_) [n:127.0.0.1:37236_    ] o.a.s.c.c.ZkStateReader Updating data for [collection1] from [0] to [1]
   [junit4]   2> 275198 INFO  (coreLoadExecutor-653-thread-1-processing-n:127.0.0.1:37236_) [n:127.0.0.1:37236_ c:collection1   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 275198 INFO  (coreLoadExecutor-653-thread-1-processing-n:127.0.0.1:37236_) [n:127.0.0.1:37236_ c:collection1   x:collection1] o.a.s.c.ZkController Check for collection zkNode:collection1
   [junit4]   2> 275198 INFO  (coreLoadExecutor-653-thread-1-processing-n:127.0.0.1:37236_) [n:127.0.0.1:37236_ c:collection1   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 275198 INFO  (coreLoadExecutor-653-thread-1-processing-n:127.0.0.1:37236_) [n:127.0.0.1:37236_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/collection1]
   [junit4]   2> 275199 INFO  (coreLoadExecutor-653-thread-1-processing-n:127.0.0.1:37236_) [n:127.0.0.1:37236_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/collection1] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 275199 INFO  (coreLoadExecutor-653-thread-1-processing-n:127.0.0.1:37236_) [n:127.0.0.1:37236_ c:collection1   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionStateFormat2Test_21913F6461EA0D1C-001/shard-1-001/cores/collection1'
   [junit4]   2> 275199 INFO  (coreLoadExecutor-653-thread-1-processing-n:127.0.0.1:37236_) [n:127.0.0.1:37236_ c:collection1   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 275199 INFO  (coreLoadExecutor-653-thread-1-processing-n:127.0.0.1:37236_) [n:127.0.0.1:37236_ c:collection1   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 275200 INFO  (coreLoadExecutor-653-thread-1-processing-n:127.0.0.1:37236_) [n:127.0.0.1:37236_ c:collection1   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 275206 INFO  (coreLoadExecutor-653-thread-1-processing-n:127.0.0.1:37236_) [n:127.0.0.1:37236_ c:collection1   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 275210 INFO  (coreLoadExecutor-653-thread-1-processing-n:127.0.0.1:37236_) [n:127.0.0.1:37236_ c:collection1   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.2.0
   [junit4]   2> 275249 INFO  (coreLoadExecutor-653-thread-1-processing-n:127.0.0.1:37236_) [n:127.0.0.1:37236_ c:collection1   x:collection1] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 275254 INFO  (coreLoadExecutor-653-thread-1-processing-n:127.0.0.1:37236_) [n:127.0.0.1:37236_ c:collection1   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 275337 INFO  (coreLoadExecutor-653-thread-1-processing-n:127.0.0.1:37236_) [n:127.0.0.1:37236_ c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 275343 INFO  (coreLoadExecutor-653-thread-1-processing-n:127.0.0.1:37236_) [n:127.0.0.1:37236_ c:collection1   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 275344 INFO  (coreLoadExecutor-653-thread-1-processing-n:127.0.0.1:37236_) [n:127.0.0.1:37236_ c:collection1   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 275358 INFO  (coreLoadExecutor-653-thread-1-processing-n:127.0.0.1:37236_) [n:127.0.0.1:37236_ c:collection1   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 275360 INFO  (coreLoadExecutor-653-thread-1-processing-n:127.0.0.1:37236_) [n:127.0.0.1:37236_ c:collection1   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 275361 INFO  (coreLoadExecutor-653-thread-1-processing-n:127.0.0.1:37236_) [n:127.0.0.1:37236_ c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 275362 INFO  (coreLoadExecutor-653-thread-1-processing-n:127.0.0.1:37236_) [n:127.0.0.1:37236_ c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 275363 INFO  (coreLoadExecutor-653-thread-1-processing-n:127.0.0.1:37236_) [n:127.0.0.1:37236_ c:collection1   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2
   [junit4]   2> 275363 INFO  (coreLoadExecutor-653-thread-1-processing-n:127.0.0.1:37236_) [n:127.0.0.1:37236_ c:collection1   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection collection1
   [junit4]   2> 275364 INFO  (coreLoadExecutor-653-thread-1-processing-n:127.0.0.1:37236_) [n:127.0.0.1:37236_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 275364 INFO  (coreLoadExecutor-653-thread-1-processing-n:127.0.0.1:37236_) [n:127.0.0.1:37236_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionStateFormat2Test_21913F6461EA0D1C-001/shard-1-001/cores/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionStateFormat2Test_21913F6461EA0D1C-001/shard-1-001/cores/collection1/data/]
   [junit4]   2> 275364 INFO  (coreLoadExecutor-653-thread-1-processing-n:127.0.0.1:37236_) [n:127.0.0.1:37236_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@5484988
   [junit4]   2> 275364 INFO  (coreLoadExecutor-653-thread-1-processing-n:127.0.0.1:37236_) [n:127.0.0.1:37236_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionStateFormat2Test_21913F6461EA0D1C-001/shard-1-001/cores/collection1/data
   [junit4]   2> 275373 INFO  (coreLoadExecutor-653-thread-1-processing-n:127.0.0.1:37236_) [n:127.0.0.1:37236_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionStateFormat2Test_21913F6461EA0D1C-001/shard-1-001/cores/collection1/data/index/
   [junit4]   2> 275373 WARN  (coreLoadExecutor-653-th

[...truncated too long message...]

ce-029/./collection1/data/index.20160723011559847 [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandler_21913F6461EA0D1C-001/solr-instance-029/./collection1/data/index.20160723011559847;done=false>>]
   [junit4]   2> 3538851 INFO  (coreCloseExecutor-12843-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory, CoreContainer#isShutdown=true
   [junit4]   2> 3538851 INFO  (coreCloseExecutor-12843-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandler_21913F6461EA0D1C-001/solr-instance-029/./collection1/data/index.20160723011559847
   [junit4]   2> 3538852 INFO  (TEST-TestReplicationHandler.doTestIndexFetchWithMasterUrl-seed#[21913F6461EA0D1C]) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@ecb40e{/solr,null,UNAVAILABLE}
   [junit4]   2> 3538853 INFO  (TEST-TestReplicationHandler.doTestIndexFetchWithMasterUrl-seed#[21913F6461EA0D1C]) [    ] o.e.j.s.ServerConnector Stopped ServerConnector@65d7490e{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
   [junit4]   2> 3538853 INFO  (TEST-TestReplicationHandler.doTestIndexFetchWithMasterUrl-seed#[21913F6461EA0D1C]) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=417193068
   [junit4]   2> 3538857 INFO  (coreCloseExecutor-12845-thread-1) [    x:collection1] o.a.s.c.SolrCore [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@e73db39
   [junit4]   2> 3538857 INFO  (coreCloseExecutor-12845-thread-1) [    x:collection1] o.a.s.u.DirectUpdateHandler2 closing DirectUpdateHandler2{commits=2,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=3,cumulative_deletesById=0,cumulative_deletesByQuery=1,cumulative_errors=0}
   [junit4]   2> 3538857 INFO  (coreCloseExecutor-12845-thread-1) [    x:collection1] o.a.s.u.SolrCoreState Closing SolrCoreState
   [junit4]   2> 3538857 INFO  (coreCloseExecutor-12845-thread-1) [    x:collection1] o.a.s.u.DefaultSolrCoreState SolrCoreState ref count has reached 0 - closing IndexWriter
   [junit4]   2> 3538857 INFO  (coreCloseExecutor-12845-thread-1) [    x:collection1] o.a.s.u.DefaultSolrCoreState closing IndexWriter with IndexWriterCloser
   [junit4]   2> 3538858 INFO  (coreCloseExecutor-12845-thread-1) [    x:collection1] o.a.s.c.SolrCore [collection1] Closing main searcher on request.
   [junit4]   2> 3538858 INFO  (coreCloseExecutor-12845-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing MockDirectoryFactory - 2 directories currently being tracked
   [junit4]   2> 3538858 INFO  (coreCloseExecutor-12845-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory looking to close /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandler_21913F6461EA0D1C-001/solr-instance-030/./collection1/data [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandler_21913F6461EA0D1C-001/solr-instance-030/./collection1/data;done=false>>]
   [junit4]   2> 3538858 INFO  (coreCloseExecutor-12845-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory, CoreContainer#isShutdown=true
   [junit4]   2> 3538858 INFO  (coreCloseExecutor-12845-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandler_21913F6461EA0D1C-001/solr-instance-030/./collection1/data
   [junit4]   2> 3538858 INFO  (coreCloseExecutor-12845-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory looking to close /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandler_21913F6461EA0D1C-001/solr-instance-030/./collection1/data/index.20160723011559564 [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandler_21913F6461EA0D1C-001/solr-instance-030/./collection1/data/index.20160723011559564;done=false>>]
   [junit4]   2> 3538858 INFO  (coreCloseExecutor-12845-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory, CoreContainer#isShutdown=true
   [junit4]   2> 3538858 INFO  (coreCloseExecutor-12845-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandler_21913F6461EA0D1C-001/solr-instance-030/./collection1/data/index.20160723011559564
   [junit4]   2> 3538859 INFO  (TEST-TestReplicationHandler.doTestIndexFetchWithMasterUrl-seed#[21913F6461EA0D1C]) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@4663cdff{/solr,null,UNAVAILABLE}
   [junit4]   2> 3538860 INFO  (SUITE-TestReplicationHandler-seed#[21913F6461EA0D1C]-worker) [    ] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandler_21913F6461EA0D1C-001
   [junit4]   2> NOTE: test params are: codec=SimpleText, sim=ClassicSimilarity, locale=id-ID, timezone=Europe/Paris
   [junit4]   2> NOTE: Linux 3.13.0-85-generic amd64/Oracle Corporation 1.8.0_74 (64-bit)/cpus=4,threads=1,free=187297928,total=526909440
   [junit4]   2> NOTE: All tests run in this JVM: [TestUseDocValuesAsStored2, TestConfigSetsAPIExclusivity, CoreSorterTest, TestJettySolrRunner, DirectUpdateHandlerOptimizeTest, TestZkChroot, TestOrdValues, JavabinLoaderTest, TestGroupingSearch, TestCollapseQParserPlugin, TestExpandComponent, TestRebalanceLeaders, TestExtendedDismaxParser, TestJsonRequest, SolrCloudExampleTest, LeaderFailoverAfterPartitionTest, TestDistribDocBasedVersion, PluginInfoTest, TestHashQParserPlugin, DirectUpdateHandlerTest, CollectionStateFormat2Test, TestPerFieldSimilarity, DistribDocExpirationUpdateProcessorTest, DistributedSuggestComponentTest, TestMissingGroups, DocValuesMissingTest, PrimitiveFieldTypeTest, DeleteInactiveReplicaTest, SparseHLLTest, ClusterStateTest, OverriddenZkACLAndCredentialsProvidersTest, HdfsUnloadDistributedZkTest, TestRuleBasedAuthorizationPlugin, XmlUpdateRequestHandlerTest, TestPostingsSolrHighlighter, TestSubQueryTransformer, TestIndexSearcher, TestCollationField, SharedFSAutoReplicaFailoverUtilsTest, UnloadDistributedZkTest, DateMathParserTest, TestSolrCLIRunExample, HardAutoCommitTest, TermVectorComponentDistributedTest, TestNamedUpdateProcessors, IgnoreCommitOptimizeUpdateProcessorFactoryTest, OverseerRolesTest, SortSpecParsingTest, TestCloudDeleteByQuery, RequestHandlersTest, TestSchemaSimilarityResource, TestSolrDynamicMBean, TestGraphMLResponseWriter, CollectionsAPISolrJTest, SolrTestCaseJ4Test, StatelessScriptUpdateProcessorFactoryTest, DistributedDebugComponentTest, TestMiniSolrCloudClusterBase, RestartWhileUpdatingTest, TestFieldCacheSort, AutoCommitTest, TestCloudSchemaless, StatsComponentTest, TestRequestStatusCollectionAPI, OverseerTest, TestSolrQueryResponse, TestXmlQParserPlugin, HdfsChaosMonkeySafeLeaderTest, InfoHandlerTest, DistributedVersionInfoTest, TestStressLiveNodes, GraphQueryTest, TestFuzzyAnalyzedSuggestions, TestSolrQueryParserDefaultOperatorResource, TestSolrIndexConfig, TestSweetSpotSimilarityFactory, HdfsBasicDistributedZk2Test, ConnectionManagerTest, TestDefaultSearchFieldResource, CircularListTest, FieldAnalysisRequestHandlerTest, TestManagedResource, TestTolerantSearch, SimpleFacetsTest, BadComponentTest, TestBlobHandler, TestPerFieldSimilarityClassic, DocValuesMultiTest, CleanupOldIndexTest, CdcrRequestHandlerTest, TestExactStatsCache, DistanceUnitsTest, MigrateRouteKeyTest, HdfsRecoverLeaseTest, TestIndexingPerformance, TestDistributedMissingSort, UpdateParamsTest, TestFieldTypeResource, AnalysisErrorHandlingTest, TestWriterPerf, ClassificationUpdateProcessorFactoryTest, SuggesterTSTTest, BasicDistributedZk2Test, CollectionsAPIDistributedZkTest, OpenCloseCoreStressTest, LeaderElectionIntegrationTest, ShardRoutingTest, RecoveryZkTest, FullSolrCloudDistribCmdsTest, TestReplicationHandler]
   [junit4] Completed [255/624 (2!)] on J2 in 522.46s, 15 tests, 1 failure, 1 skipped <<< FAILURES!

[...truncated 5825 lines...]
ERROR: Connection was broken: java.io.IOException: Unexpected termination of the channel
	at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:50)
Caused by: java.io.EOFException
	at java.io.ObjectInputStream$PeekInputStream.readFully(ObjectInputStream.java:2325)
	at java.io.ObjectInputStream$BlockDataInputStream.readShort(ObjectInputStream.java:2794)
	at java.io.ObjectInputStream.readStreamHeader(ObjectInputStream.java:801)
	at java.io.ObjectInputStream.<init>(ObjectInputStream.java:299)
	at hudson.remoting.ObjectInputStreamEx.<init>(ObjectInputStreamEx.java:48)
	at hudson.remoting.AbstractSynchronousByteArrayCommandTransport.read(AbstractSynchronousByteArrayCommandTransport.java:34)
	at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:48)

Build step 'Invoke Ant' marked build as failure
ERROR: Step ‘Archive the artifacts’ failed: no workspace for Lucene-Solr-NightlyTests-6.x #126
ERROR: Step ‘Publish JUnit test result report’ failed: no workspace for Lucene-Solr-NightlyTests-6.x #126
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
ERROR: lucene is offline; cannot locate latest1.8
ERROR: lucene is offline; cannot locate latest1.8




Re: [JENKINS] Lucene-Solr-NightlyTests-6.x - Build # 125 - Failure

Posted by Nicholas Knize <nk...@gmail.com>.
pushed fix for TestDoubleRangeFieldQueries.

On Thu, Jul 21, 2016 at 2:54 PM, Apache Jenkins Server <
jenkins@builds.apache.org> wrote:

> Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-6.x/125/
>
> 6 tests failed.
> FAILED:  org.apache.lucene.index.TestDuelingCodecsAtNight.testBigEquals
>
> Error Message:
> Java heap space
>
> Stack Trace:
> java.lang.OutOfMemoryError: Java heap space
>         at
> __randomizedtesting.SeedInfo.seed([62829676FE994519:8186CDF35582815E]:0)
>         at
> org.apache.lucene.util.fst.BytesStore.writeBytes(BytesStore.java:112)
>         at
> org.apache.lucene.util.fst.ByteSequenceOutputs.write(ByteSequenceOutputs.java:120)
>         at
> org.apache.lucene.util.fst.ByteSequenceOutputs.write(ByteSequenceOutputs.java:35)
>         at
> org.apache.lucene.util.fst.Outputs.writeFinalOutput(Outputs.java:60)
>         at org.apache.lucene.util.fst.FST.pack(FST.java:1714)
>         at org.apache.lucene.util.fst.Builder.finish(Builder.java:500)
>         at
> org.apache.lucene.codecs.memory.MemoryPostingsFormat$TermsWriter.finish(MemoryPostingsFormat.java:267)
>         at
> org.apache.lucene.codecs.memory.MemoryPostingsFormat$MemoryFieldsConsumer.write(MemoryPostingsFormat.java:401)
>         at
> org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsWriter.write(PerFieldPostingsFormat.java:198)
>         at
> org.apache.lucene.codecs.FieldsConsumer.merge(FieldsConsumer.java:105)
>         at
> org.apache.lucene.index.SegmentMerger.mergeTerms(SegmentMerger.java:216)
>         at
> org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:101)
>         at
> org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4312)
>         at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3889)
>         at
> org.apache.lucene.index.SerialMergeScheduler.merge(SerialMergeScheduler.java:40)
>         at
> org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:2052)
>         at
> org.apache.lucene.index.IndexWriter.doAfterSegmentFlushed(IndexWriter.java:4953)
>         at
> org.apache.lucene.index.DocumentsWriter$MergePendingEvent.process(DocumentsWriter.java:742)
>         at
> org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:4991)
>         at
> org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:4982)
>         at
> org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1565)
>         at
> org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1307)
>         at
> org.apache.lucene.index.RandomIndexWriter.addDocument(RandomIndexWriter.java:171)
>         at
> org.apache.lucene.index.TestDuelingCodecs.createRandomIndex(TestDuelingCodecs.java:139)
>         at
> org.apache.lucene.index.TestDuelingCodecsAtNight.testBigEquals(TestDuelingCodecsAtNight.java:34)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>         at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>         at java.lang.reflect.Method.invoke(Method.java:498)
>         at
> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
>         at
> com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
>         at
> com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
>
>
> FAILED:
> org.apache.lucene.search.TestDoubleRangeFieldQueries.testMultiValued
>
> Error Message:
> wrong hit (first of possibly more):  FAIL: id=204 should not match but
> did  queryBox=Box(Infinity TO Infinity)  box=Box(Infinity TO Infinity)
> queryType=CONTAINS  deleted?=false
>
> Stack Trace:
> java.lang.AssertionError: wrong hit (first of possibly more):
>
> FAIL: id=204 should not match but did
>  queryBox=Box(Infinity TO Infinity)
>  box=Box(Infinity TO Infinity)
>  queryType=CONTAINS
>  deleted?=false
>         at
> __randomizedtesting.SeedInfo.seed([AE24BFDDECD216AC:7A04DBEF221056E4]:0)
>         at org.junit.Assert.fail(Assert.java:93)
>         at
> org.apache.lucene.search.BaseRangeFieldQueryTestCase.verify(BaseRangeFieldQueryTestCase.java:278)
>         at
> org.apache.lucene.search.BaseRangeFieldQueryTestCase.doTestRandom(BaseRangeFieldQueryTestCase.java:154)
>         at
> org.apache.lucene.search.BaseRangeFieldQueryTestCase.testMultiValued(BaseRangeFieldQueryTestCase.java:73)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>         at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>         at java.lang.reflect.Method.invoke(Method.java:498)
>         at
> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
>         at
> com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
>         at
> com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
>         at
> com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
>         at
> org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
>         at
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
>         at
> org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
>         at
> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
>         at
> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
>         at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>         at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
>         at
> com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
>         at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
>         at
> com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
>         at
> com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
>         at
> com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
>         at
> com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
>         at
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
>         at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>         at
> org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
>         at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
>         at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
>         at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>         at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>         at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>         at
> org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
>         at
> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
>         at
> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
>         at
> org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
>         at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>         at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
>         at java.lang.Thread.run(Thread.java:745)
>
>
> FAILED:  org.apache.lucene.search.TestDoubleRangeFieldQueries.testRandomBig
>
> Error Message:
> wrong hit (first of possibly more):  FAIL: id=49 should not match but did
> queryBox=Box(Infinity TO Infinity)  box=Box(Infinity TO Infinity)
> queryType=CONTAINS  deleted?=false
>
> Stack Trace:
> java.lang.AssertionError: wrong hit (first of possibly more):
>
> FAIL: id=49 should not match but did
>  queryBox=Box(Infinity TO Infinity)
>  box=Box(Infinity TO Infinity)
>  queryType=CONTAINS
>  deleted?=false
>         at
> __randomizedtesting.SeedInfo.seed([AE24BFDDECD216AC:2973C2527D8B6A2C]:0)
>         at org.junit.Assert.fail(Assert.java:93)
>         at
> org.apache.lucene.search.BaseRangeFieldQueryTestCase.verify(BaseRangeFieldQueryTestCase.java:278)
>         at
> org.apache.lucene.search.BaseRangeFieldQueryTestCase.doTestRandom(BaseRangeFieldQueryTestCase.java:154)
>         at
> org.apache.lucene.search.BaseRangeFieldQueryTestCase.testRandomBig(BaseRangeFieldQueryTestCase.java:69)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>         at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>         at java.lang.reflect.Method.invoke(Method.java:498)
>         at
> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
>         at
> com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
>         at
> com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
>         at
> com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
>         at
> org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
>         at
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
>         at
> org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
>         at
> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
>         at
> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
>         at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>         at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
>         at
> com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
>         at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
>         at
> com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
>         at
> com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
>         at
> com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
>         at
> com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
>         at
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
>         at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>         at
> org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
>         at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
>         at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
>         at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>         at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>         at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>         at
> org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
>         at
> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
>         at
> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
>         at
> org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
>         at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>         at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
>         at java.lang.Thread.run(Thread.java:745)
>
>
> FAILED:
> org.apache.solr.search.stats.TestDistribIDF.testMultiCollectionQuery
>
> Error Message:
> Doc1 score=0.9808292 Doc2 score=0.9808292 expected:<1> but was:<0>
>
> Stack Trace:
> java.lang.AssertionError: Doc1 score=0.9808292 Doc2 score=0.9808292
> expected:<1> but was:<0>
>         at
> __randomizedtesting.SeedInfo.seed([1F45A3F87F67AC9C:E3664C9A311A9E0]:0)
>         at org.junit.Assert.fail(Assert.java:93)
>         at org.junit.Assert.failNotEquals(Assert.java:647)
>         at org.junit.Assert.assertEquals(Assert.java:128)
>         at org.junit.Assert.assertEquals(Assert.java:472)
>         at
> org.apache.solr.search.stats.TestDistribIDF.testMultiCollectionQuery(TestDistribIDF.java:183)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>         at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>         at java.lang.reflect.Method.invoke(Method.java:498)
>         at
> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
>         at
> com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
>         at
> com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
>         at
> com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
>         at
> com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
>         at
> org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
>         at
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
>         at
> org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
>         at
> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
>         at
> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
>         at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>         at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
>         at
> com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
>         at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
>         at
> com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
>         at
> com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
>         at
> com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
>         at
> com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
>         at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>         at
> com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
>         at
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
>         at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>         at
> org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
>         at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
>         at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
>         at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>         at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>         at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>         at
> org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
>         at
> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
>         at
> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
>         at
> org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
>         at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>         at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
>         at java.lang.Thread.run(Thread.java:745)
>
>
> FAILED:
> junit.framework.TestSuite.org.apache.solr.handler.TestReplicationHandler
>
> Error Message:
> ObjectTracker found 2 object(s) that were not released!!!
> [NRTCachingDirectory, NRTCachingDirectory]
>
> Stack Trace:
> java.lang.AssertionError: ObjectTracker found 2 object(s) that were not
> released!!! [NRTCachingDirectory, NRTCachingDirectory]
>         at __randomizedtesting.SeedInfo.seed([1F45A3F87F67AC9C]:0)
>         at org.junit.Assert.fail(Assert.java:93)
>         at org.junit.Assert.assertTrue(Assert.java:43)
>         at org.junit.Assert.assertNull(Assert.java:551)
>         at
> org.apache.solr.SolrTestCaseJ4.teardownTestCases(SolrTestCaseJ4.java:258)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>         at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>         at java.lang.reflect.Method.invoke(Method.java:498)
>         at
> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
>         at
> com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:834)
>         at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>         at
> com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
>         at
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
>         at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>         at
> org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
>         at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
>         at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
>         at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>         at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>         at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>         at
> org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
>         at
> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
>         at
> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
>         at
> org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
>         at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>         at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
>         at java.lang.Thread.run(Thread.java:745)
>
>
> FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test
>
> Error Message:
> Timeout occured while waiting response from server at:
> http://127.0.0.1:36062
>
> Stack Trace:
> org.apache.solr.client.solrj.SolrServerException: Timeout occured while
> waiting response from server at: http://127.0.0.1:36062
>         at
> org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:601)
>         at
> org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:259)
>         at
> org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:248)
>         at
> org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
>         at
> org.apache.solr.cloud.CollectionsAPIDistributedZkTest.makeRequest(CollectionsAPIDistributedZkTest.java:399)
>         at
> org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testErrorHandling(CollectionsAPIDistributedZkTest.java:515)
>         at
> org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test(CollectionsAPIDistributedZkTest.java:179)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>         at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>         at java.lang.reflect.Method.invoke(Method.java:498)
>         at
> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
>         at
> com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
>         at
> com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
>         at
> com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
>         at
> org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
>         at
> org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
>         at
> com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
>         at
> org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
>         at
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
>         at
> org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
>         at
> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
>         at
> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
>         at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>         at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
>         at
> com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
>         at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
>         at
> com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
>         at
> com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
>         at
> com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
>         at
> com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
>         at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>         at
> com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
>         at
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
>         at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>         at
> org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
>         at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
>         at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
>         at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>         at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>         at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>         at
> org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
>         at
> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
>         at
> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
>         at
> org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
>         at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>         at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
>         at java.lang.Thread.run(Thread.java:745)
> Caused by: java.net.SocketTimeoutException: Read timed out
>         at java.net.SocketInputStream.socketRead0(Native Method)
>         at
> java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
>         at java.net.SocketInputStream.read(SocketInputStream.java:170)
>         at java.net.SocketInputStream.read(SocketInputStream.java:141)
>         at
> org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:160)
>         at
> org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:84)
>         at
> org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:273)
>         at
> org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
>         at
> org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
>         at
> org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
>         at
> org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)
>         at
> org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)
>         at
> org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197)
>         at
> org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)
>         at
> org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)
>         at
> org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685)
>         at
> org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487)
>         at
> org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
>         at
> org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
>         at
> org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
>         at
> org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
>         at
> org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:495)
>         ... 47 more
>
>
>
>
> Build Log:
> [...truncated 1725 lines...]
>    [junit4] Suite: org.apache.lucene.index.TestDuelingCodecsAtNight
>    [junit4]   2> NOTE: download the large Jenkins line-docs file by
> running 'ant get-jenkins-line-docs' in the lucene directory.
>    [junit4]   2> NOTE: reproduce with: ant test
> -Dtestcase=TestDuelingCodecsAtNight -Dtests.method=testBigEquals
> -Dtests.seed=62829676FE994519 -Dtests.multiplier=2 -Dtests.nightly=true
> -Dtests.slow=true
> -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt
> -Dtests.locale=ar-AE -Dtests.timezone=America/Managua -Dtests.asserts=true
> -Dtests.file.encoding=US-ASCII
>    [junit4] ERROR   2423s J1 | TestDuelingCodecsAtNight.testBigEquals <<<
>    [junit4]    > Throwable #1: java.lang.OutOfMemoryError: Java heap space
>    [junit4]    >        at
> __randomizedtesting.SeedInfo.seed([62829676FE994519:8186CDF35582815E]:0)
>    [junit4]    >        at
> org.apache.lucene.util.fst.BytesStore.writeBytes(BytesStore.java:112)
>    [junit4]    >        at
> org.apache.lucene.util.fst.ByteSequenceOutputs.write(ByteSequenceOutputs.java:120)
>    [junit4]    >        at
> org.apache.lucene.util.fst.ByteSequenceOutputs.write(ByteSequenceOutputs.java:35)
>    [junit4]    >        at
> org.apache.lucene.util.fst.Outputs.writeFinalOutput(Outputs.java:60)
>    [junit4]    >        at
> org.apache.lucene.util.fst.FST.pack(FST.java:1714)
>    [junit4]    >        at
> org.apache.lucene.util.fst.Builder.finish(Builder.java:500)
>    [junit4]    >        at
> org.apache.lucene.codecs.memory.MemoryPostingsFormat$TermsWriter.finish(MemoryPostingsFormat.java:267)
>    [junit4]    >        at
> org.apache.lucene.codecs.memory.MemoryPostingsFormat$MemoryFieldsConsumer.write(MemoryPostingsFormat.java:401)
>    [junit4]    >        at
> org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsWriter.write(PerFieldPostingsFormat.java:198)
>    [junit4]    >        at
> org.apache.lucene.codecs.FieldsConsumer.merge(FieldsConsumer.java:105)
>    [junit4]    >        at
> org.apache.lucene.index.SegmentMerger.mergeTerms(SegmentMerger.java:216)
>    [junit4]    >        at
> org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:101)
>    [junit4]    >        at
> org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4312)
>    [junit4]    >        at
> org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3889)
>    [junit4]    >        at
> org.apache.lucene.index.SerialMergeScheduler.merge(SerialMergeScheduler.java:40)
>    [junit4]    >        at
> org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:2052)
>    [junit4]    >        at
> org.apache.lucene.index.IndexWriter.doAfterSegmentFlushed(IndexWriter.java:4953)
>    [junit4]    >        at
> org.apache.lucene.index.DocumentsWriter$MergePendingEvent.process(DocumentsWriter.java:742)
>    [junit4]    >        at
> org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:4991)
>    [junit4]    >        at
> org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:4982)
>    [junit4]    >        at
> org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1565)
>    [junit4]    >        at
> org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1307)
>    [junit4]    >        at
> org.apache.lucene.index.RandomIndexWriter.addDocument(RandomIndexWriter.java:171)
>    [junit4]    >        at
> org.apache.lucene.index.TestDuelingCodecs.createRandomIndex(TestDuelingCodecs.java:139)
>    [junit4]    >        at
> org.apache.lucene.index.TestDuelingCodecsAtNight.testBigEquals(TestDuelingCodecsAtNight.java:34)
>    [junit4]   2> NOTE: leaving temporary files on disk at:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/test/J1/temp/lucene.index.TestDuelingCodecsAtNight_62829676FE994519-001
>    [junit4]   2> NOTE: test params are: codec=Asserting(Lucene62): {},
> docValues:{}, maxPointsInLeafNode=1581, maxMBSortInHeap=6.757363644236303,
> sim=RandomSimilarity(queryNorm=true,coord=yes):
> {titleTokenized=DFI(Standardized), body=DFR I(n)L1}, locale=ar-AE,
> timezone=America/Managua
>    [junit4]   2> NOTE: Linux 3.13.0-85-generic amd64/Oracle Corporation
> 1.8.0_74 (64-bit)/cpus=4,threads=1,free=269638936,total=405274624
>    [junit4]   2> NOTE: All tests run in this JVM: [TestTermdocPerf,
> TestLucene60PointsFormat, TestInfoStream, TestPackedInts, TestTransactions,
> TestBooleanRewrites, TestAttributeSource, TestFilterIterator,
> TestDocIDMerger, TestRegExp, TestPolygon2D, TestParallelLeafReader,
> TestAddIndexes, TestSpanExplanations, TestPositiveScoresOnlyCollector,
> TestDirectPacked, TestSetOnce, TestVirtualMethod, TestSloppyPhraseQuery2,
> TestFixedBitSet, TestTimSorter, TestVersion,
> TestAllFilesHaveChecksumFooter, TestCharsRef, TestDoc, TestAssertions,
> TestSleepingLockWrapper, TestSPIClassIterator, TestBufferedChecksum,
> TestRegexpRandom, TestHighCompressionMode, TestRegexpQuery,
> TestIndexCommit, TestReadOnlyIndex, TestTermVectorsWriter,
> TestParallelTermEnum, TestFieldMaskingSpanQuery, TestPerSegmentDeletes,
> TestNewestSegment, TestScorerPerf, TestTwoPhaseCommitTool,
> TestGeoEncodingUtils, TestIndexWriterExceptions, TestSearchForDuplicates,
> TestSimpleSearchEquivalence, TestTermVectors, TestDocument,
> TestFileSwitchDirectory, TestBasics, TestIndexSorting,
> TestPositionIncrement, TestApproximationSearchEquivalence,
> TestIndexSearcher, Test2BNumericDocValues, TestBinaryDocument,
> TestLockFactory, TestDocIdsWriter, TestPerFieldPostingsFormat,
> TestSnapshotDeletionPolicy, TestSortRescorer, TestPostingsOffsets,
> TestSegmentReader, MultiCollectorTest, TestStopFilter,
> TestLucene50TermVectorsFormat, TestSpans, TestNIOFSDirectory,
> TestFieldReuse, TestSparseFixedBitSet, TestDocumentsWriterDeleteQueue,
> TestBinaryTerms, TestFilterSpans, TestThreadedForceMerge,
> TestPerFieldDocValuesFormat, TestIndexWriterReader, TestIndexWriterMerging,
> TestFSTs, TestIndexWriterWithThreads, TestMultiMMap, TestDeletionPolicy,
> TestFlushByRamOrCountsPolicy, TestIndexWriterMergePolicy,
> TestDuelingCodecs, TestBoolean2, TestAtomicUpdate, TestStressAdvance,
> TestNorms, TestIndexWriterCommit, TestDirectoryReaderReopen,
> TestTimeLimitingCollector, TestSearchWithThreads, TestSearchAfter,
> TestDocumentsWriterStallControl, TestTopDocsMerge, TestIndexWriterUnicode,
> TestNRTReaderWithThreads, TestTieredMergePolicy, TestLevenshteinAutomata,
> TestByteSlices, TestTermsEnum, TestToken, TestCustomSearcherSort,
> TestHugeRamFile, TestLegacyNumericUtils, TestTransactionRollback, TestFlex,
> TestUTF32ToUTF8, TestTermRangeQuery, TestMultiTermConstantScore,
> TestFuzzyQuery, TestExternalCodecs, TestPrefixInBooleanQuery,
> TestComplexExplanationsOfNonMatches, TestUnicodeUtil, TestPrefixCodedTerms,
> TestSumDocFreq, TestConstantScoreQuery, TestElevationComparator,
> TestIndexWriterConfig, TestMultiTermQueryRewrites, TestSimilarity,
> TestRollingBuffer, TestFilterLeafReader, TestCachingCollector,
> TestRamUsageEstimator, TestNoMergeScheduler, TestPhrasePrefixQuery,
> TestCharFilter, TestCloseableThreadLocal, TestScoreCachingWrappingScorer,
> TestIntsRef, TestIOUtils, Test2BPagedBytes, TestCharArraySet,
> TestWordlistLoader, TestPackedTokenAttributeImpl, TestBlockPostingsFormat,
> TestGeoUtils, TestDocInverterPerFieldErrorInfo, TestDuelingCodecsAtNight]
>    [junit4] Completed [435/435 (1!)] on J1 in 2466.59s, 3 tests, 1 error
> <<< FAILURES!
>
> [...truncated 1 lines...]
>    [junit4] JVM J1: stdout was not empty, see:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/test/temp/junit4-J1-20160721_111954_897.sysout
>    [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
>    [junit4] java.lang.OutOfMemoryError: Java heap space
>    [junit4] Dumping heap to
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/heapdumps/java_pid27116.hprof
> ...
>    [junit4] Heap dump file created [422506181 bytes in 0.847 secs]
>    [junit4] <<< JVM J1: EOF ----
>
> [...truncated 7226 lines...]
>    [junit4] Suite: org.apache.lucene.search.TestDoubleRangeFieldQueries
>    [junit4]   2> NOTE: download the large Jenkins line-docs file by
> running 'ant get-jenkins-line-docs' in the lucene directory.
>    [junit4]   2> NOTE: reproduce with: ant test
> -Dtestcase=TestDoubleRangeFieldQueries -Dtests.method=testMultiValued
> -Dtests.seed=AE24BFDDECD216AC -Dtests.multiplier=2 -Dtests.nightly=true
> -Dtests.slow=true
> -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt
> -Dtests.locale=hu -Dtests.timezone=Europe/San_Marino -Dtests.asserts=true
> -Dtests.file.encoding=US-ASCII
>    [junit4] FAILURE 4.20s J0 | TestDoubleRangeFieldQueries.testMultiValued
> <<<
>    [junit4]    > Throwable #1: java.lang.AssertionError: wrong hit (first
> of possibly more):
>    [junit4]    > FAIL: id=204 should not match but did
>    [junit4]    >  queryBox=Box(Infinity TO Infinity)
>    [junit4]    >  box=Box(Infinity TO Infinity)
>    [junit4]    >  queryType=CONTAINS
>    [junit4]    >  deleted?=false
>    [junit4]    >        at
> __randomizedtesting.SeedInfo.seed([AE24BFDDECD216AC:7A04DBEF221056E4]:0)
>    [junit4]    >        at
> org.apache.lucene.search.BaseRangeFieldQueryTestCase.verify(BaseRangeFieldQueryTestCase.java:278)
>    [junit4]    >        at
> org.apache.lucene.search.BaseRangeFieldQueryTestCase.doTestRandom(BaseRangeFieldQueryTestCase.java:154)
>    [junit4]    >        at
> org.apache.lucene.search.BaseRangeFieldQueryTestCase.testMultiValued(BaseRangeFieldQueryTestCase.java:73)
>    [junit4]    >        at java.lang.Thread.run(Thread.java:745)
>    [junit4]   2> NOTE: download the large Jenkins line-docs file by
> running 'ant get-jenkins-line-docs' in the lucene directory.
>    [junit4]   2> NOTE: reproduce with: ant test
> -Dtestcase=TestDoubleRangeFieldQueries -Dtests.method=testRandomBig
> -Dtests.seed=AE24BFDDECD216AC -Dtests.multiplier=2 -Dtests.nightly=true
> -Dtests.slow=true
> -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt
> -Dtests.locale=hu -Dtests.timezone=Europe/San_Marino -Dtests.asserts=true
> -Dtests.file.encoding=US-ASCII
>    [junit4] FAILURE 35.1s J0 | TestDoubleRangeFieldQueries.testRandomBig
> <<<
>    [junit4]    > Throwable #1: java.lang.AssertionError: wrong hit (first
> of possibly more):
>    [junit4]    > FAIL: id=49 should not match but did
>    [junit4]    >  queryBox=Box(Infinity TO Infinity)
>    [junit4]    >  box=Box(Infinity TO Infinity)
>    [junit4]    >  queryType=CONTAINS
>    [junit4]    >  deleted?=false
>    [junit4]    >        at
> __randomizedtesting.SeedInfo.seed([AE24BFDDECD216AC:2973C2527D8B6A2C]:0)
>    [junit4]    >        at
> org.apache.lucene.search.BaseRangeFieldQueryTestCase.verify(BaseRangeFieldQueryTestCase.java:278)
>    [junit4]    >        at
> org.apache.lucene.search.BaseRangeFieldQueryTestCase.doTestRandom(BaseRangeFieldQueryTestCase.java:154)
>    [junit4]    >        at
> org.apache.lucene.search.BaseRangeFieldQueryTestCase.testRandomBig(BaseRangeFieldQueryTestCase.java:69)
>    [junit4]    >        at java.lang.Thread.run(Thread.java:745)
>    [junit4]   2> NOTE: leaving temporary files on disk at:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/sandbox/test/J0/temp/lucene.search.TestDoubleRangeFieldQueries_AE24BFDDECD216AC-001
>    [junit4]   2> NOTE: test params are: codec=Asserting(Lucene62):
> {id=PostingsFormat(name=Memory doPackFST= false)},
> docValues:{id=DocValuesFormat(name=Memory)}, maxPointsInLeafNode=1844,
> maxMBSortInHeap=6.78445457997637, sim=ClassicSimilarity, locale=hu,
> timezone=Europe/San_Marino
>    [junit4]   2> NOTE: Linux 3.13.0-85-generic amd64/Oracle Corporation
> 1.8.0_74 (64-bit)/cpus=4,threads=1,free=179129064,total=341311488
>    [junit4]   2> NOTE: All tests run in this JVM: [FuzzyLikeThisQueryTest,
> TestDocValuesTermsQuery, TestTermAutomatonQuery, TestLatLonPoint,
> TestBigIntegerPoint, TestDoubleRangeField, TestHalfFloatPoint,
> TestInetAddressPoint, TestLatLonDocValuesField, TestPayloadSpanUtil,
> TestDoubleRangeFieldQueries]
>    [junit4] Completed [18/20 (1!)] on J0 in 42.60s, 5 tests, 2 failures
> <<< FAILURES!
>
> [...truncated 2751 lines...]
>    [junit4] Suite: org.apache.solr.search.stats.TestDistribIDF
>    [junit4]   2> Creating dataDir:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.search.stats.TestDistribIDF_1F45A3F87F67AC9C-001/init-core-data-001
>    [junit4]   2> 1078911 INFO
> (SUITE-TestDistribIDF-seed#[1F45A3F87F67AC9C]-worker) [    ]
> o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true) via:
> @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN,
> clientAuth=NaN)
>    [junit4]   2> 1078915 INFO
> (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [
> ] o.a.s.SolrTestCaseJ4 ###Starting testMultiCollectionQuery
>    [junit4]   2> 1078915 INFO
> (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [
> ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
>    [junit4]   2> 1078917 INFO  (Thread-1456) [    ] o.a.s.c.ZkTestServer
> client port:0.0.0.0/0.0.0.0:0
>    [junit4]   2> 1078917 INFO  (Thread-1456) [    ] o.a.s.c.ZkTestServer
> Starting server
>    [junit4]   2> 1079017 INFO
> (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [
> ] o.a.s.c.ZkTestServer start zk server on port:52388
>    [junit4]   2> 1079017 INFO
> (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [
> ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
>    [junit4]   2> 1079018 INFO
> (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [
> ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
>    [junit4]   2> 1079025 INFO  (zkCallback-2741-thread-1) [    ]
> o.a.s.c.c.ConnectionManager Watcher
> org.apache.solr.common.cloud.ConnectionManager@64cee6ba
> name:ZooKeeperConnection Watcher:127.0.0.1:52388 got event WatchedEvent
> state:SyncConnected type:None path:null path:null type:None
>    [junit4]   2> 1079026 INFO
> (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [
> ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
>    [junit4]   2> 1079026 INFO
> (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [
> ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
>    [junit4]   2> 1079026 INFO
> (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [
> ] o.a.s.c.c.SolrZkClient makePath: /solr/solr.xml
>    [junit4]   2> 1079041 INFO  (jetty-launcher-2740-thread-1) [    ]
> o.e.j.s.Server jetty-9.3.8.v20160314
>    [junit4]   2> 1079041 INFO  (jetty-launcher-2740-thread-2) [    ]
> o.e.j.s.Server jetty-9.3.8.v20160314
>    [junit4]   2> 1079042 INFO  (jetty-launcher-2740-thread-3) [    ]
> o.e.j.s.Server jetty-9.3.8.v20160314
>    [junit4]   2> 1079043 INFO  (jetty-launcher-2740-thread-1) [    ]
> o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@52f75917
> {/solr,null,AVAILABLE}
>    [junit4]   2> 1079045 INFO  (jetty-launcher-2740-thread-1) [    ]
> o.e.j.s.ServerConnector Started ServerConnector@216f0751
> {HTTP/1.1,[http/1.1]}{127.0.0.1:54619}
>    [junit4]   2> 1079045 INFO  (jetty-launcher-2740-thread-1) [    ]
> o.e.j.s.Server Started @1083810ms
>    [junit4]   2> 1079045 INFO  (jetty-launcher-2740-thread-1) [    ]
> o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr,
> hostPort=54619}
>    [junit4]   2> 1079045 INFO  (jetty-launcher-2740-thread-1) [    ]
> o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init():
> sun.misc.Launcher$AppClassLoader@73d16e93
>    [junit4]   2> 1079045 INFO  (jetty-launcher-2740-thread-1) [    ]
> o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory:
> '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.search.stats.TestDistribIDF_1F45A3F87F67AC9C-001/tempDir-001/node1'
>    [junit4]   2> 1079045 INFO  (jetty-launcher-2740-thread-1) [    ]
> o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
>    [junit4]   2> 1079045 INFO  (jetty-launcher-2740-thread-1) [    ]
> o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find
> system property or JNDI)
>    [junit4]   2> 1079045 INFO  (jetty-launcher-2740-thread-1) [    ]
> o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
>    [junit4]   2> 1079047 INFO  (jetty-launcher-2740-thread-1) [    ]
> o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
>    [junit4]   2> 1079049 INFO  (jetty-launcher-2740-thread-2) [    ]
> o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5e041f38
> {/solr,null,AVAILABLE}
>    [junit4]   2> 1079049 INFO  (jetty-launcher-2740-thread-2) [    ]
> o.e.j.s.ServerConnector Started ServerConnector@347ecf61
> {HTTP/1.1,[http/1.1]}{127.0.0.1:49958}
>    [junit4]   2> 1079049 INFO  (jetty-launcher-2740-thread-2) [    ]
> o.e.j.s.Server Started @1083815ms
>    [junit4]   2> 1079049 INFO  (jetty-launcher-2740-thread-2) [    ]
> o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr,
> hostPort=49958}
>    [junit4]   2> 1079049 INFO  (jetty-launcher-2740-thread-2) [    ]
> o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init():
> sun.misc.Launcher$AppClassLoader@73d16e93
>    [junit4]   2> 1079049 INFO  (jetty-launcher-2740-thread-2) [    ]
> o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory:
> '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.search.stats.TestDistribIDF_1F45A3F87F67AC9C-001/tempDir-001/node2'
>    [junit4]   2> 1079049 INFO  (jetty-launcher-2740-thread-2) [    ]
> o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
>    [junit4]   2> 1079049 INFO  (jetty-launcher-2740-thread-2) [    ]
> o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find
> system property or JNDI)
>    [junit4]   2> 1079050 INFO  (jetty-launcher-2740-thread-2) [    ]
> o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
>    [junit4]   2> 1079051 INFO  (jetty-launcher-2740-thread-2) [    ]
> o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
>    [junit4]   2> 1079052 INFO  (jetty-launcher-2740-thread-3) [    ]
> o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@31a6e920
> {/solr,null,AVAILABLE}
>    [junit4]   2> 1079052 INFO  (jetty-launcher-2740-thread-3) [    ]
> o.e.j.s.ServerConnector Started ServerConnector@7d06c7f9
> {HTTP/1.1,[http/1.1]}{127.0.0.1:46930}
>    [junit4]   2> 1079052 INFO  (jetty-launcher-2740-thread-3) [    ]
> o.e.j.s.Server Started @1083818ms
>    [junit4]   2> 1079052 INFO  (jetty-launcher-2740-thread-3) [    ]
> o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr,
> hostPort=46930}
>    [junit4]   2> 1079053 INFO  (jetty-launcher-2740-thread-3) [    ]
> o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init():
> sun.misc.Launcher$AppClassLoader@73d16e93
>    [junit4]   2> 1079053 INFO  (jetty-launcher-2740-thread-3) [    ]
> o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory:
> '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.search.stats.TestDistribIDF_1F45A3F87F67AC9C-001/tempDir-001/node3'
>    [junit4]   2> 1079053 INFO  (jetty-launcher-2740-thread-3) [    ]
> o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
>    [junit4]   2> 1079053 INFO  (jetty-launcher-2740-thread-3) [    ]
> o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find
> system property or JNDI)
>    [junit4]   2> 1079053 INFO  (jetty-launcher-2740-thread-3) [    ]
> o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
>    [junit4]   2> 1079067 INFO  (jetty-launcher-2740-thread-3) [    ]
> o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
>    [junit4]   2> 1079071 INFO  (zkCallback-2742-thread-1) [    ]
> o.a.s.c.c.ConnectionManager Watcher
> org.apache.solr.common.cloud.ConnectionManager@6e1c60a2
> name:ZooKeeperConnection Watcher:127.0.0.1:52388/solr got event
> WatchedEvent state:SyncConnected type:None path:null path:null type:None
>    [junit4]   2> 1079071 INFO  (jetty-launcher-2740-thread-1) [    ]
> o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
>    [junit4]   2> 1079072 INFO  (jetty-launcher-2740-thread-1) [    ]
> o.a.s.c.c.SolrZkClient Using default ZkACLProvider
>    [junit4]   2> 1079072 INFO  (jetty-launcher-2740-thread-1) [    ]
> o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
>    [junit4]   2> 1079091 INFO  (zkCallback-2743-thread-1) [    ]
> o.a.s.c.c.ConnectionManager Watcher
> org.apache.solr.common.cloud.ConnectionManager@ecde4e4
> name:ZooKeeperConnection Watcher:127.0.0.1:52388/solr got event
> WatchedEvent state:SyncConnected type:None path:null path:null type:None
>    [junit4]   2> 1079092 INFO  (jetty-launcher-2740-thread-1) [    ]
> o.a.s.c.CorePropertiesLocator Config-defined core root directory:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.search.stats.TestDistribIDF_1F45A3F87F67AC9C-001/tempDir-001/node1/.
>    [junit4]   2> 1079092 INFO  (jetty-launcher-2740-thread-2) [    ]
> o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
>    [junit4]   2> 1079092 INFO  (jetty-launcher-2740-thread-1) [    ]
> o.a.s.c.CoreContainer New CoreContainer 1005926444
>    [junit4]   2> 1079092 INFO  (jetty-launcher-2740-thread-1) [    ]
> o.a.s.c.CoreContainer Loading cores into CoreContainer
> [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.search.stats.TestDistribIDF_1F45A3F87F67AC9C-001/tempDir-001/node1]
>    [junit4]   2> 1079092 INFO  (jetty-launcher-2740-thread-2) [    ]
> o.a.s.c.c.SolrZkClient Using default ZkACLProvider
>    [junit4]   2> 1079092 WARN  (jetty-launcher-2740-thread-1) [    ]
> o.a.s.c.CoreContainer Couldn't add files from
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.search.stats.TestDistribIDF_1F45A3F87F67AC9C-001/tempDir-001/node1/lib
> to classpath:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.search.stats.TestDistribIDF_1F45A3F87F67AC9C-001/tempDir-001/node1/lib
>    [junit4]   2> 1079092 INFO  (jetty-launcher-2740-thread-1) [    ]
> o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout :
> 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost :
> 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize :
> 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy :
> false,useRetries : false,connectionsEvictorSleepDelay :
> 5000,maxConnectionIdleTime : 40000,
>    [junit4]   2> 1079093 INFO  (jetty-launcher-2740-thread-1) [    ]
> o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with
> params: socketTimeout=340000&connTimeout=45000&retry=true
>    [junit4]   2> 1079093 INFO  (jetty-launcher-2740-thread-2) [    ]
> o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
>    [junit4]   2> 1079092 INFO  (zkCallback-2744-thread-1) [    ]
> o.a.s.c.c.ConnectionManager Watcher
> org.apache.solr.common.cloud.ConnectionManager@158ebf96
> name:ZooKeeperConnection Watcher:127.0.0.1:52388/solr got event
> WatchedEvent state:SyncConnected type:None path:null path:null type:None
>    [junit4]   2> 1079094 INFO  (jetty-launcher-2740-thread-1) [    ]
> o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
>    [junit4]   2> 1079094 INFO  (jetty-launcher-2740-thread-3) [    ]
> o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
>    [junit4]   2> 1079094 INFO  (jetty-launcher-2740-thread-1) [    ]
> o.a.s.l.LogWatcher Registering Log Listener [Log4j
> (org.slf4j.impl.Log4jLoggerFactory)]
>    [junit4]   2> 1079094 INFO  (jetty-launcher-2740-thread-1) [    ]
> o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:52388/solr
>    [junit4]   2> 1079094 INFO  (jetty-launcher-2740-thread-3) [    ]
> o.a.s.c.c.SolrZkClient Using default ZkACLProvider
>    [junit4]   2> 1079094 INFO  (jetty-launcher-2740-thread-1) [    ]
> o.a.s.c.ZkController zkHost includes chroot
>    [junit4]   2> 1079094 INFO  (jetty-launcher-2740-thread-1) [    ]
> o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
>    [junit4]   2> 1079095 INFO  (jetty-launcher-2740-thread-3) [    ]
> o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
>    [junit4]   2> 1079096 INFO  (jetty-launcher-2740-thread-1) [    ]
> o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
>    [junit4]   2> 1079096 INFO  (zkCallback-2747-thread-1) [    ]
> o.a.s.c.c.ConnectionManager Watcher
> org.apache.solr.common.cloud.ConnectionManager@7d1f4e95
> name:ZooKeeperConnection Watcher:127.0.0.1:52388 got event WatchedEvent
> state:SyncConnected type:None path:null path:null type:None
>    [junit4]   2> 1079096 INFO  (jetty-launcher-2740-thread-1) [    ]
> o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
>    [junit4]   2> 1079096 INFO  (jetty-launcher-2740-thread-1) [    ]
> o.a.s.c.c.SolrZkClient Using default ZkACLProvider
>    [junit4]   2> 1079097 INFO  (jetty-launcher-2740-thread-1)
> [n:127.0.0.1:54619_solr    ] o.a.s.c.ZkController Added new OnReconnect
> listener org.apache.solr.cloud.ZkController$$Lambda$73/369477848@5c1b8b00
>    [junit4]   2> 1079098 INFO  (jetty-launcher-2740-thread-1)
> [n:127.0.0.1:54619_solr    ] o.a.s.c.c.ConnectionManager Waiting for
> client to connect to ZooKeeper
>    [junit4]   2> 1079099 INFO
> (zkCallback-2748-thread-1-processing-n:127.0.0.1:54619_solr)
> [n:127.0.0.1:54619_solr    ] o.a.s.c.c.ConnectionManager Watcher
> org.apache.solr.common.cloud.ConnectionManager@70735b96
> name:ZooKeeperConnection Watcher:127.0.0.1:52388/solr got event
> WatchedEvent state:SyncConnected type:None path:null path:null type:None
>    [junit4]   2> 1079099 INFO  (jetty-launcher-2740-thread-1)
> [n:127.0.0.1:54619_solr    ] o.a.s.c.c.ConnectionManager Client is
> connected to ZooKeeper
>    [junit4]   2> 1079100 INFO  (jetty-launcher-2740-thread-1)
> [n:127.0.0.1:54619_solr    ] o.a.s.c.c.SolrZkClient makePath:
> /overseer/queue
>    [junit4]   2> 1079101 INFO  (jetty-launcher-2740-thread-2) [    ]
> o.a.s.c.CorePropertiesLocator Config-defined core root directory:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.search.stats.TestDistribIDF_1F45A3F87F67AC9C-001/tempDir-001/node2/.
>    [junit4]   2> 1079101 INFO  (jetty-launcher-2740-thread-2) [    ]
> o.a.s.c.CoreContainer New CoreContainer 1804464227
>    [junit4]   2> 1079101 INFO  (jetty-launcher-2740-thread-2) [    ]
> o.a.s.c.CoreContainer Loading cores into CoreContainer
> [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.search.stats.TestDistribIDF_1F45A3F87F67AC9C-001/tempDir-001/node2]
>    [junit4]   2> 1079101 WARN  (jetty-launcher-2740-thread-2) [    ]
> o.a.s.c.CoreContainer Couldn't add files from
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.search.stats.TestDistribIDF_1F45A3F87F67AC9C-001/tempDir-001/node2/lib
> to classpath:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.search.stats.TestDistribIDF_1F45A3F87F67AC9C-001/tempDir-001/node2/lib
>    [junit4]   2> 1079101 INFO  (jetty-launcher-2740-thread-2) [    ]
> o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout :
> 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost :
> 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize :
> 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy :
> false,useRetries : false,connectionsEvictorSleepDelay :
> 5000,maxConnectionIdleTime : 40000,
>    [junit4]   2> 1079101 INFO  (jetty-launcher-2740-thread-1)
> [n:127.0.0.1:54619_solr    ] o.a.s.c.c.SolrZkClient makePath:
> /overseer/collection-queue-work
>    [junit4]   2> 1079103 INFO  (jetty-launcher-2740-thread-1)
> [n:127.0.0.1:54619_solr    ] o.a.s.c.c.SolrZkClient makePath:
> /overseer/collection-map-running
>    [junit4]   2> 1079104 INFO  (jetty-launcher-2740-thread-1)
> [n:127.0.0.1:54619_solr    ] o.a.s.c.c.SolrZkClient makePath:
> /overseer/collection-map-completed
>    [junit4]   2> 1079106 INFO  (jetty-launcher-2740-thread-1)
> [n:127.0.0.1:54619_solr    ] o.a.s.c.c.SolrZkClient makePath:
> /overseer/collection-map-failure
>    [junit4]   2> 1079106 INFO  (jetty-launcher-2740-thread-2) [    ]
> o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with
> params: socketTimeout=340000&connTimeout=45000&retry=true
>    [junit4]   2> 1079106 INFO  (jetty-launcher-2740-thread-3) [    ]
> o.a.s.c.CorePropertiesLocator Config-defined core root directory:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.search.stats.TestDistribIDF_1F45A3F87F67AC9C-001/tempDir-001/node3/.
>    [junit4]   2> 1079106 INFO  (jetty-launcher-2740-thread-3) [    ]
> o.a.s.c.CoreContainer New CoreContainer 1499331252
>    [junit4]   2> 1079106 INFO  (jetty-launcher-2740-thread-3) [    ]
> o.a.s.c.CoreContainer Loading cores into CoreContainer
> [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.search.stats.TestDistribIDF_1F45A3F87F67AC9C-001/tempDir-001/node3]
>    [junit4]   2> 1079106 WARN  (jetty-launcher-2740-thread-3) [    ]
> o.a.s.c.CoreContainer Couldn't add files from
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.search.stats.TestDistribIDF_1F45A3F87F67AC9C-001/tempDir-001/node3/lib
> to classpath:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.search.stats.TestDistribIDF_1F45A3F87F67AC9C-001/tempDir-001/node3/lib
>    [junit4]   2> 1079108 INFO  (jetty-launcher-2740-thread-3) [    ]
> o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout :
> 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost :
> 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize :
> 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy :
> false,useRetries : false,connectionsEvictorSleepDelay :
> 5000,maxConnectionIdleTime : 40000,
>    [junit4]   2> 1079108 INFO  (jetty-launcher-2740-thread-3) [    ]
> o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with
> params: socketTimeout=340000&connTimeout=45000&retry=true
>    [junit4]   2> 1079109 INFO  (jetty-launcher-2740-thread-3) [    ]
> o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
>    [junit4]   2> 1079109 INFO  (jetty-launcher-2740-thread-3) [    ]
> o.a.s.l.LogWatcher Registering Log Listener [Log4j
> (org.slf4j.impl.Log4jLoggerFactory)]
>    [junit4]   2> 1079109 INFO  (jetty-launcher-2740-thread-3) [    ]
> o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:52388/solr
>    [junit4]   2> 1079109 INFO  (jetty-launcher-2740-thread-3) [    ]
> o.a.s.c.ZkController zkHost includes chroot
>    [junit4]   2> 1079109 INFO  (jetty-launcher-2740-thread-3) [    ]
> o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
>    [junit4]   2> 1079109 INFO  (jetty-launcher-2740-thread-1)
> [n:127.0.0.1:54619_solr    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
>    [junit4]   2> 1079106 INFO  (jetty-launcher-2740-thread-2) [    ]
> o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
>    [junit4]   2> 1079110 INFO  (jetty-launcher-2740-thread-2) [    ]
> o.a.s.l.LogWatcher Registering Log Listener [Log4j
> (org.slf4j.impl.Log4jLoggerFactory)]
>    [junit4]   2> 1079110 INFO  (jetty-launcher-2740-thread-2) [    ]
> o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:52388/solr
>    [junit4]   2> 1079110 INFO  (jetty-launcher-2740-thread-2) [    ]
> o.a.s.c.ZkController zkHost includes chroot
>    [junit4]   2> 1079111 INFO  (jetty-launcher-2740-thread-2) [    ]
> o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
>    [junit4]   2> 1079110 INFO  (jetty-launcher-2740-thread-3) [    ]
> o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
>    [junit4]   2> 1079111 INFO  (jetty-launcher-2740-thread-2) [    ]
> o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
>    [junit4]   2> 1079111 INFO  (zkCallback-2754-thread-1) [    ]
> o.a.s.c.c.ConnectionManager Watcher
> org.apache.solr.common.cloud.ConnectionManager@15b9783a
> name:ZooKeeperConnection Watcher:127.0.0.1:52388 got event WatchedEvent
> state:SyncConnected type:None path:null path:null type:None
>    [junit4]   2> 1079112 INFO  (jetty-launcher-2740-thread-3) [    ]
> o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
>    [junit4]   2> 1079112 INFO  (jetty-launcher-2740-thread-3) [    ]
> o.a.s.c.c.SolrZkClient Using default ZkACLProvider
>    [junit4]   2> 1079112 INFO  (zkCallback-2755-thread-1) [    ]
> o.a.s.c.c.ConnectionManager Watcher
> org.apache.solr.common.cloud.ConnectionManager@48ecfc7d
> name:ZooKeeperConnection Watcher:127.0.0.1:52388 got event WatchedEvent
> state:SyncConnected type:None path:null path:null type:None
>    [junit4]   2> 1079112 INFO  (jetty-launcher-2740-thread-1)
> [n:127.0.0.1:54619_solr    ] o.a.s.c.c.SolrZkClient makePath: /collections
>    [junit4]   2> 1079113 INFO  (jetty-launcher-2740-thread-2) [    ]
> o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
>    [junit4]   2> 1079113 INFO  (jetty-launcher-2740-thread-2) [    ]
> o.a.s.c.c.SolrZkClient Using default ZkACLProvider
>    [junit4]   2> 1079113 INFO  (jetty-launcher-2740-thread-3)
> [n:127.0.0.1:46930_solr    ] o.a.s.c.ZkController Added new OnReconnect
> listener org.apache.solr.cloud.ZkController$$Lambda$73/369477848@2d43ea88
>    [junit4]   2> 1079114 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0)
> [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
>    [junit4]   2> EndOfStreamException: Unable to read additional data from
> client sessionid 0x1560dfdace20007, likely client has closed socket
>    [junit4]   2>        at
> org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
>    [junit4]   2>        at
> org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
>    [junit4]   2>        at java.lang.Thread.run(Thread.java:745)
>    [junit4]   2> 1079114 INFO  (jetty-launcher-2740-thread-1)
> [n:127.0.0.1:54619_solr    ] o.a.s.c.c.SolrZkClient makePath:
> /aliases.json
>    [junit4]   2> 1079115 INFO  (jetty-launcher-2740-thread-3)
> [n:127.0.0.1:46930_solr    ] o.a.s.c.c.ConnectionManager Waiting for
> client to connect to ZooKeeper
>    [junit4]   2> 1079115 INFO  (jetty-launcher-2740-thread-2)
> [n:127.0.0.1:49958_solr    ] o.a.s.c.ZkController Added new OnReconnect
> listener org.apache.solr.cloud.ZkController$$Lambda$73/369477848@4495a7b4
>    [junit4]   2> 1079116 INFO  (jetty-launcher-2740-thread-1)
> [n:127.0.0.1:54619_solr    ] o.a.s.c.c.SolrZkClient makePath:
> /clusterstate.json
>    [junit4]   2> 1079116 INFO  (jetty-launcher-2740-thread-2)
> [n:127.0.0.1:49958_solr    ] o.a.s.c.c.ConnectionManager Waiting for
> client to connect to ZooKeeper
>    [junit4]   2> 1079117 INFO
> (zkCallback-2756-thread-1-processing-n:127.0.0.1:46930_solr)
> [n:127.0.0.1:46930_solr    ] o.a.s.c.c.ConnectionManager Watcher
> org.apache.solr.common.cloud.ConnectionManager@5492d1b8
> name:ZooKeeperConnection Watcher:127.0.0.1:52388/solr got event
> WatchedEvent state:SyncConnected type:None path:null path:null type:None
>    [junit4]   2> 1079117 INFO  (jetty-launcher-2740-thread-1)
> [n:127.0.0.1:54619_solr    ] o.a.s.c.c.SolrZkClient makePath:
> /security.json
>    [junit4]   2> 1079117 INFO  (jetty-launcher-2740-thread-3)
> [n:127.0.0.1:46930_solr    ] o.a.s.c.c.ConnectionManager Client is
> connected to ZooKeeper
>    [junit4]   2> 1079117 INFO
> (zkCallback-2757-thread-1-processing-n:127.0.0.1:49958_solr)
> [n:127.0.0.1:49958_solr    ] o.a.s.c.c.ConnectionManager Watcher
> org.apache.solr.common.cloud.ConnectionManager@efed6
> name:ZooKeeperConnection Watcher:127.0.0.1:52388/solr got event
> WatchedEvent state:SyncConnected type:None path:null path:null type:None
>    [junit4]   2> 1079117 INFO  (jetty-launcher-2740-thread-2)
> [n:127.0.0.1:49958_solr    ] o.a.s.c.c.ConnectionManager Client is
> connected to ZooKeeper
>    [junit4]   2> 1079118 INFO  (jetty-launcher-2740-thread-1)
> [n:127.0.0.1:54619_solr    ] o.a.s.c.c.ZkStateReader Updating cluster
> state from ZooKeeper...
>    [junit4]   2> 1079119 INFO  (jetty-launcher-2740-thread-1)
> [n:127.0.0.1:54619_solr    ] o.a.s.c.c.ZkStateReader Loaded empty cluster
> properties
>    [junit4]   2> 1079120 INFO  (jetty-launcher-2740-thread-1)
> [n:127.0.0.1:54619_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes
> from ZooKeeper... (0) -> (0)
>    [junit4]   2> 1079123 INFO  (jetty-launcher-2740-thread-1)
> [n:127.0.0.1:54619_solr    ] o.a.s.c.c.SolrZkClient makePath:
> /overseer_elect
>    [junit4]   2> 1079123 INFO  (jetty-launcher-2740-thread-2)
> [n:127.0.0.1:49958_solr    ] o.a.s.c.c.ZkStateReader Updating cluster
> state from ZooKeeper...
>    [junit4]   2> 1079124 INFO  (jetty-launcher-2740-thread-2)
> [n:127.0.0.1:49958_solr    ] o.a.s.c.c.ZkStateReader Loaded empty cluster
> properties
>    [junit4]   2> 1079124 INFO  (jetty-launcher-2740-thread-2)
> [n:127.0.0.1:49958_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes
> from ZooKeeper... (0) -> (0)
>    [junit4]   2> 1079124 INFO  (jetty-launcher-2740-thread-3)
> [n:127.0.0.1:46930_solr    ] o.a.s.c.c.ZkStateReader Updating cluster
> state from ZooKeeper...
>    [junit4]   2> 1079125 INFO  (jetty-launcher-2740-thread-1)
> [n:127.0.0.1:54619_solr    ] o.a.s.c.c.SolrZkClient makePath:
> /overseer_elect/election
>    [junit4]   2> 1079125 INFO  (jetty-launcher-2740-thread-3)
> [n:127.0.0.1:46930_solr    ] o.a.s.c.c.ZkStateReader Loaded empty cluster
> properties
>    [junit4]   2> 1079125 INFO  (jetty-launcher-2740-thread-3)
> [n:127.0.0.1:46930_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes
> from ZooKeeper... (0) -> (0)
>    [junit4]   2> 1079126 INFO  (jetty-launcher-2740-thread-1)
> [n:127.0.0.1:54619_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
>    [junit4]   2> 1079126 INFO  (jetty-launcher-2740-thread-1)
> [n:127.0.0.1:54619_solr    ] o.a.s.c.LeaderElector Joined leadership
> election with path:
> /overseer_elect/election/96279825213423621-127.0.0.1:54619_solr-n_0000000000
>    [junit4]   2> 1079127 INFO  (jetty-launcher-2740-thread-1)
> [n:127.0.0.1:54619_solr    ] o.a.s.c.OverseerElectionContext I am going
> to be the leader 127.0.0.1:54619_solr
>    [junit4]   2> 1079127 INFO  (jetty-launcher-2740-thread-1)
> [n:127.0.0.1:54619_solr    ] o.a.s.c.c.SolrZkClient makePath:
> /overseer_elect/leader
>    [junit4]   2> 1079128 INFO  (jetty-launcher-2740-thread-1)
> [n:127.0.0.1:54619_solr    ] o.a.s.c.Overseer Overseer
> (id=96279825213423621-127.0.0.1:54619_solr-n_0000000000) starting
>    [junit4]   2> 1079128 INFO  (jetty-launcher-2740-thread-2)
> [n:127.0.0.1:49958_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
>    [junit4]   2> 1079128 INFO  (jetty-launcher-2740-thread-2)
> [n:127.0.0.1:49958_solr    ] o.a.s.c.LeaderElector Joined leadership
> election with path:
> /overseer_elect/election/96279825213423625-127.0.0.1:49958_solr-n_0000000001
>    [junit4]   2> 1079129 INFO  (jetty-launcher-2740-thread-3)
> [n:127.0.0.1:46930_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
>    [junit4]   2> 1079129 INFO  (jetty-launcher-2740-thread-2)
> [n:127.0.0.1:49958_solr    ] o.a.s.c.LeaderElector Watching path
> /overseer_elect/election/96279825213423621-127.0.0.1:54619_solr-n_0000000000
> to know if I could be the leader
>    [junit4]   2> 1079129 INFO  (jetty-launcher-2740-thread-1)
> [n:127.0.0.1:54619_solr    ] o.a.s.c.c.SolrZkClient makePath:
> /overseer/queue-work
>    [junit4]   2> 1079130 INFO  (jetty-launcher-2740-thread-3)
> [n:127.0.0.1:46930_solr    ] o.a.s.c.LeaderElector Joined leadership
> election with path:
> /overseer_elect/election/96279825213423624-127.0.0.1:46930_solr-n_0000000002
>    [junit4]   2> 1079130 INFO  (jetty-launcher-2740-thread-2)
> [n:127.0.0.1:49958_solr    ] o.a.s.c.ZkController Register node as live
> in ZooKeeper:/live_nodes/127.0.0.1:49958_solr
>    [junit4]   2> 1079130 INFO  (jetty-launcher-2740-thread-2)
> [n:127.0.0.1:49958_solr    ] o.a.s.c.c.SolrZkClient makePath:
> /live_nodes/127.0.0.1:49958_solr
>    [junit4]   2> 1079130 INFO  (jetty-launcher-2740-thread-3)
> [n:127.0.0.1:46930_solr    ] o.a.s.c.LeaderElector Watching path
> /overseer_elect/election/96279825213423625-127.0.0.1:49958_solr-n_0000000001
> to know if I could be the leader
>    [junit4]   2> 1079131 INFO
> (zkCallback-2756-thread-1-processing-n:127.0.0.1:46930_solr)
> [n:127.0.0.1:46930_solr    ] o.a.s.c.c.ZkStateReader A live node change:
> [WatchedEvent state:SyncConnected type:NodeChildrenChanged
> path:/live_nodes], has occurred - updating... (live nodes size: [0])
>    [junit4]   2> 1079131 INFO
> (zkCallback-2748-thread-1-processing-n:127.0.0.1:54619_solr)
> [n:127.0.0.1:54619_solr    ] o.a.s.c.c.ZkStateReader A live node change:
> [WatchedEvent state:SyncConnected type:NodeChildrenChanged
> path:/live_nodes], has occurred - updating... (live nodes size: [0])
>    [junit4]   2> 1079131 INFO
> (zkCallback-2757-thread-1-processing-n:127.0.0.1:49958_solr)
> [n:127.0.0.1:49958_solr    ] o.a.s.c.c.ZkStateReader A live node change:
> [WatchedEvent state:SyncConnected type:NodeChildrenChanged
> path:/live_nodes], has occurred - updating... (live nodes size: [0])
>    [junit4]   2> 1079131 INFO  (jetty-launcher-2740-thread-3)
> [n:127.0.0.1:46930_solr    ] o.a.s.c.ZkController Publish
> node=127.0.0.1:46930_solr as DOWN
>    [junit4]   2> 1079132 INFO
> (zkCallback-2748-thread-1-processing-n:127.0.0.1:54619_solr)
> [n:127.0.0.1:54619_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes
> from ZooKeeper... (0) -> (1)
>    [junit4]   2> 1079132 INFO
> (zkCallback-2756-thread-1-processing-n:127.0.0.1:46930_solr)
> [n:127.0.0.1:46930_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes
> from ZooKeeper... (0) -> (1)
>    [junit4]   2> 1079134 INFO  (jetty-launcher-2740-thread-3)
> [n:127.0.0.1:46930_solr    ] o.a.s.c.ZkController Register node as live
> in ZooKeeper:/live_nodes/127.0.0.1:46930_solr
>    [junit4]   2> 1079134 INFO  (jetty-launcher-2740-thread-3)
> [n:127.0.0.1:46930_solr    ] o.a.s.c.c.SolrZkClient makePath:
> /live_nodes/127.0.0.1:46930_solr
>    [junit4]   2> 1079135 INFO
> (zkCallback-2748-thread-1-processing-n:127.0.0.1:54619_solr)
> [n:127.0.0.1:54619_solr    ] o.a.s.c.c.ZkStateReader A live node change:
> [WatchedEvent state:SyncConnected type:NodeChildrenChanged
> path:/live_nodes], has occurred - updating... (live nodes size: [1])
>    [junit4]   2> 1079135 INFO
> (zkCallback-2756-thread-1-processing-n:127.0.0.1:46930_solr)
> [n:127.0.0.1:46930_solr    ] o.a.s.c.c.ZkStateReader A live node change:
> [WatchedEvent state:SyncConnected type:NodeChildrenChanged
> path:/live_nodes], has occurred - updating... (live nodes size: [1])
>    [junit4]   2> 1079135 INFO
> (zkCallback-2748-thread-1-processing-n:127.0.0.1:54619_solr)
> [n:127.0.0.1:54619_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes
> from ZooKeeper... (1) -> (2)
>    [junit4]   2> 1079136 INFO
> (zkCallback-2756-thread-1-processing-n:127.0.0.1:46930_solr)
> [n:127.0.0.1:46930_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes
> from ZooKeeper... (1) -> (2)
>    [junit4]   2> 1079155 INFO
> (zkCallback-2757-thread-1-processing-n:127.0.0.1:49958_solr)
> [n:127.0.0.1:49958_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes
> from ZooKeeper... (0) -> (1)
>    [junit4]   2> 1079156 INFO  (jetty-launcher-2740-thread-3)
> [n:127.0.0.1:46930_solr    ] o.a.s.c.CoreContainer Security conf doesn't
> exist. Skipping setup for authorization module.
>    [junit4]   2> 1079156 INFO  (jetty-launcher-2740-thread-3)
> [n:127.0.0.1:46930_solr    ] o.a.s.c.CoreContainer No authentication
> plugin used.
>    [junit4]   2> 1079157 INFO  (jetty-launcher-2740-thread-3)
> [n:127.0.0.1:46930_solr    ] o.a.s.c.CorePropertiesLocator Looking for
> core definitions underneath
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.search.stats.TestDistribIDF_1F45A3F87F67AC9C-001/tempDir-001/node3/.
>    [junit4]   2> 1079157 INFO  (jetty-launcher-2740-thread-3)
> [n:127.0.0.1:46930_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core
> definitions
>    [junit4]   2> 1079157 INFO
> (zkCallback-2757-thread-2-processing-n:127.0.0.1:49958_solr)
> [n:127.0.0.1:49958_solr    ] o.a.s.c.c.ZkStateReader A live node change:
> [WatchedEvent state:SyncConnected type:NodeChildrenChanged
> path:/live_nodes], has occurred - updating... (live nodes size: [1])
>    [junit4]   2> 1079158 INFO
> (zkCallback-2757-thread-2-processing-n:127.0.0.1:49958_solr)
> [n:127.0.0.1:49958_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes
> from ZooKeeper... (1) -> (2)
>    [junit4]   2> 1079158 INFO  (jetty-launcher-2740-thread-3)
> [n:127.0.0.1:46930_solr    ] o.a.s.s.SolrDispatchFilter
> user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0
>    [junit4]   2> 1079158 INFO  (jetty-launcher-2740-thread-3)
> [n:127.0.0.1:46930_solr    ] o.a.s.s.SolrDispatchFilter
> SolrDispatchFilter.init() done
>    [junit4]   2> 1079159 INFO  (jetty-launcher-2740-thread-1)
> [n:127.0.0.1:54619_solr    ] o.a.s.c.OverseerAutoReplicaFailoverThread
> Starting OverseerAutoReplicaFailoverThread
> autoReplicaFailoverWorkLoopDelay=10000
> autoReplicaFailoverWaitAfterExpiration=30000
> autoReplicaFailoverBadNodeExpiration=60000
>    [junit4]   2> 1079160 INFO
> (OverseerStateUpdate-96279825213423621-127.0.0.1:54619_solr-n_0000000000)
> [n:127.0.0.1:54619_solr    ] o.a.s.c.Overseer Starting to work on the
> main queue
>    [junit4]   2> 1079160 INFO
> (OverseerStateUpdate-96279825213423621-127.0.0.1:54619_solr-n_0000000000)
> [n:127.0.0.1:54619_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes
> from ZooKeeper... (2) -> (2)
>    [junit4]   2> 1079161 INFO
> (OverseerStateUpdate-96279825213423621-127.0.0.1:54619_solr-n_0000000000)
> [n:127.0.0.1:54619_solr    ] o.a.s.c.Overseer processMessage: queueSize:
> 1, message = {
>    [junit4]   2>   "operation":"downnode",
>    [junit4]   2>   "node_name":"127.0.0.1:46930_solr"} current state
> version: 0
>    [junit4]   2> 1079161 INFO
> (OverseerStateUpdate-96279825213423621-127.0.0.1:54619_solr-n_0000000000)
> [n:127.0.0.1:54619_solr    ] o.a.s.c.o.NodeMutator DownNode state invoked
> for node: 127.0.0.1:46930_solr
>    [junit4]   2> 1079163 INFO
> (OverseerCollectionConfigSetProcessor-96279825213423621-127.0.0.1:54619_solr-n_0000000000)
> [n:127.0.0.1:54619_solr    ] o.a.s.c.OverseerTaskProcessor Process
> current queue of overseer operations
>    [junit4]   2> 1079171 INFO  (jetty-launcher-2740-thread-2)
> [n:127.0.0.1:49958_solr    ] o.a.s.c.CoreContainer Security conf doesn't
> exist. Skipping setup for authorization module.
>    [junit4]   2> 1079171 INFO  (jetty-launcher-2740-thread-2)
> [n:127.0.0.1:49958_solr    ] o.a.s.c.CoreContainer No authentication
> plugin used.
>    [junit4]   2> 1079171 INFO  (jetty-launcher-2740-thread-1)
> [n:127.0.0.1:54619_solr    ] o.a.s.c.ZkController Publish
> node=127.0.0.1:54619_solr as DOWN
>    [junit4]   2> 1079172 INFO  (jetty-launcher-2740-thread-1)
> [n:127.0.0.1:54619_solr    ] o.a.s.c.ZkController Register node as live
> in ZooKeeper:/live_nodes/127.0.0.1:54619_solr
>    [junit4]   2> 1079173 INFO  (jetty-launcher-2740-thread-1)
> [n:127.0.0.1:54619_solr    ] o.a.s.c.c.SolrZkClient makePath:
> /live_nodes/127.0.0.1:54619_solr
>    [junit4]   2> 1079173 INFO  (jetty-launcher-2740-thread-2)
> [n:127.0.0.1:49958_solr    ] o.a.s.c.CorePropertiesLocator Looking for
> core definitions underneath
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.search.stats.TestDistribIDF_1F45A3F87F67AC9C-001/tempDir-001/node2/.
>    [junit4]   2> 1079173 INFO
> (zkCallback-2756-thread-1-processing-n:127.0.0.1:46930_solr)
> [n:127.0.0.1:46930_solr    ] o.a.s.c.c.ZkStateReader A live node change:
> [WatchedEvent state:SyncConnected type:NodeChildrenChanged
> path:/live_nodes], has occurred - updating... (live nodes size: [2])
>    [junit4]   2> 1079173 INFO
> (zkCallback-2757-thread-2-processing-n:127.0.0.1:49958_solr)
> [n:127.0.0.1:49958_solr    ] o.a.s.c.c.ZkStateReader A live node change:
> [WatchedEvent state:SyncConnected type:NodeChildrenChanged
> path:/live_nodes], has occurred - updating... (live nodes size: [2])
>    [junit4]   2> 1079173 INFO
> (zkCallback-2748-thread-1-processing-n:127.0.0.1:54619_solr)
> [n:127.0.0.1:54619_solr    ] o.a.s.c.c.ZkStateReader A live node change:
> [WatchedEvent state:SyncConnected type:NodeChildrenChanged
> path:/live_nodes], has occurred - updating... (live nodes size: [2])
>    [junit4]   2> 1079174 INFO
> (zkCallback-2757-thread-2-processing-n:127.0.0.1:49958_solr)
> [n:127.0.0.1:49958_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes
> from ZooKeeper... (2) -> (3)
>    [junit4]   2> 1079174 INFO
> (zkCallback-2748-thread-1-processing-n:127.0.0.1:54619_solr)
> [n:127.0.0.1:54619_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes
> from ZooKeeper... (2) -> (3)
>    [junit4]   2> 1079173 INFO
> (OverseerStateUpdate-96279825213423621-127.0.0.1:54619_solr-n_0000000000)
> [n:127.0.0.1:54619_solr    ] o.a.s.c.Overseer processMessage: queueSize:
> 1, message = {
>    [junit4]   2>   "operation":"downnode",
>    [junit4]   2>   "node_name":"127.0.0.1:54619_solr"} current state
> version: 0
>    [junit4]   2> 1079174 INFO
> (OverseerStateUpdate-96279825213423621-127.0.0.1:54619_solr-n_0000000000)
> [n:127.0.0.1:54619_solr    ] o.a.s.c.o.NodeMutator DownNode state invoked
> for node: 127.0.0.1:54619_solr
>    [junit4]   2> 1079178 INFO
> (zkCallback-2756-thread-1-processing-n:127.0.0.1:46930_solr)
> [n:127.0.0.1:46930_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes
> from ZooKeeper... (2) -> (3)
>    [junit4]   2> 1079180 INFO  (jetty-launcher-2740-thread-2)
> [n:127.0.0.1:49958_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core
> definitions
>    [junit4]   2> 1079181 INFO  (jetty-launcher-2740-thread-2)
> [n:127.0.0.1:49958_solr    ] o.a.s.s.SolrDispatchFilter
> user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0
>    [junit4]   2> 1079181 INFO  (jetty-launcher-2740-thread-2)
> [n:127.0.0.1:49958_solr    ] o.a.s.s.SolrDispatchFilter
> SolrDispatchFilter.init() done
>    [junit4]   2> 1079208 INFO  (jetty-launcher-2740-thread-1)
> [n:127.0.0.1:54619_solr    ] o.a.s.c.CoreContainer Security conf doesn't
> exist. Skipping setup for authorization module.
>    [junit4]   2> 1079208 INFO  (jetty-launcher-2740-thread-1)
> [n:127.0.0.1:54619_solr    ] o.a.s.c.CoreContainer No authentication
> plugin used.
>    [junit4]   2> 1079209 INFO  (jetty-launcher-2740-thread-1)
> [n:127.0.0.1:54619_solr    ] o.a.s.c.CorePropertiesLocator Looking for
> core definitions underneath
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.search.stats.TestDistribIDF_1F45A3F87F67AC9C-001/tempDir-001/node1/.
>    [junit4]   2> 1079209 INFO  (jetty-launcher-2740-thread-1)
> [n:127.0.0.1:54619_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core
> definitions
>    [junit4]   2> 1079210 INFO  (jetty-launcher-2740-thread-1)
> [n:127.0.0.1:54619_solr    ] o.a.s.s.SolrDispatchFilter
> user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0
>    [junit4]   2> 1079210 INFO  (jetty-launcher-2740-thread-1)
> [n:127.0.0.1:54619_solr    ] o.a.s.s.SolrDispatchFilter
> SolrDispatchFilter.init() done
>    [junit4]   2> 1079210 INFO
> (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [
> ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
>    [junit4]   2> 1079214 INFO
> (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [
> ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
>    [junit4]   2> 1079214 INFO  (zkCallback-2760-thread-1) [    ]
> o.a.s.c.c.ConnectionManager Watcher
> org.apache.solr.common.cloud.ConnectionManager@613281a3
> name:ZooKeeperConnection Watcher:127.0.0.1:52388 got event WatchedEvent
> state:SyncConnected type:None path:null path:null type:None
>    [junit4]   2> 1079214 INFO
> (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [
> ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
>    [junit4]   2> 1079215 INFO
> (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [
> ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
>    [junit4]   2> 1079216 INFO
> (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [
> ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
>    [junit4]   2> 1079217 INFO
> (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [
> ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
>    [junit4]   2> 1079217 INFO  (zkCallback-2762-thread-1) [    ]
> o.a.s.c.c.ConnectionManager Watcher
> org.apache.solr.common.cloud.ConnectionManager@18785709
> name:ZooKeeperConnection Watcher:127.0.0.1:52388/solr got event
> WatchedEvent state:SyncConnected type:None path:null path:null type:None
>    [junit4]   2> 1079217 INFO
> (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [
> ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
>    [junit4]   2> 1079217 INFO
> (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [
> ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
>    [junit4]   2> 1079218 INFO
> (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [
> ] o.a.s.c.c.SolrZkClient makePath:
> /configs/conf1/solrconfig-spellcheckcomponent.xml
>    [junit4]   2> 1079220 INFO
> (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [
> ] o.a.s.c.c.SolrZkClient makePath:
> /configs/conf1/schema-null-charfilters-analyzer.xml
>    [junit4]   2> 1079221 INFO
> (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [
> ] o.a.s.c.c.SolrZkClient makePath:
> /configs/conf1/regex-boost-processor-test.txt
>    [junit4]   2> 1079222 INFO
> (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [
> ] o.a.s.c.c.SolrZkClient makePath:
> /configs/conf1/solrconfig-spellchecker.xml
>    [junit4]   2> 1079223 INFO
> (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [
> ] o.a.s.c.c.SolrZkClient makePath:
> /configs/conf1/bad-schema-multiple-asterisk-copyfield-dest-should-fail-test.xml
>    [junit4]   2> 1079225 INFO
> (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [
> ] o.a.s.c.c.SolrZkClient makePath:
> /configs/conf1/schema-docValuesFaceting.xml
>    [junit4]   2> 1079226 INFO
> (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [
> ] o.a.s.c.c.SolrZkClient makePath:
> /configs/conf1/solrconfig-tlog-with-delayingcomponent.xml
>    [junit4]   2> 1079227 INFO
> (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [
> ] o.a.s.c.c.SolrZkClient makePath:
> /configs/conf1/bad-schema-currency-ft-bogus-default-code.xml
>    [junit4]   2> 1079228 INFO
> (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [
> ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema-numeric.xml
>    [junit4]   2> 1079229 INFO
> (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [
> ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig_codec.xml
>    [junit4]   2> 1079230 INFO
> (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [
> ] o.a.s.c.c.SolrZkClient makePath:
> /configs/conf1/bad-schema-multiple-asterisk-copyfield-source-should-fail-test.xml
>    [junit4]   2> 1079231 INFO
> (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [
> ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema_latest.xml
>    [junit4]   2> 1079233 INFO
> (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [
> ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema-sorts.xml
>    [junit4]   2> 1079234 INFO
> (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [
> ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig-headers.xml
>    [junit4]   2> 1079235 INFO
> (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [
> ] o.a.s.c.c.SolrZkClient makePath:
> /configs/conf1/solrconfig-implicitproperties.xml
>    [junit4]   2> 1079236 INFO
> (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [
> ] o.a.s.c.c.SolrZkClient makePath:
> /configs/conf1/bad-solrconfig-bogus-scriptengine-name.xml
>    [junit4]   2> 1079237 INFO
> (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [
> ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema-snippet-types.incl
>    [junit4]   2> 1079238 INFO
> (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [
> ] o.a.s.c.c.SolrZkClient makePath:
> /configs/conf1/solrconfig-parsing-update-processor-chains.xml
>    [junit4]   2> 1079240 INFO
> (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [
> ] o.a.s.c.c.SolrZkClient makePath:
> /configs/conf1/schema-docValuesMissing.xml
>    [junit4]   2> 1079241 INFO
> (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [
> ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig-noopregen.xml
>    [junit4]   2> 1079242 INFO
> (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [
> ] o.a.s.c.c.SolrZkClient makePath:
> /configs/conf1/schema-one-field-no-dynamic-field-unique-key.xml
>    [junit4]   2> 1079243 INFO
> (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [
> ] o.a.s.c.c.SolrZkClient makePath:
> /configs/conf1/solrconfig-script-updateprocessor.xml
>    [junit4]   2> 1079244 INFO
> (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [
> ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/bad-error-solrconfig.xml
>    [junit4]   2> 1079245 INFO
> (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [
> ] o.a.s.c.c.SolrZkClient makePath:
> /configs/conf1/solrconfig-suggestercomponent.xml
>    [junit4]   2> 1079246 INFO
> (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [
> ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/bad-currency.xml
>    [junit4]   2> 1079247 INFO
> (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [
> ] o.a.s.c.c.SolrZkClient makePath:
> /configs/conf1/solrconfig-query-parser-init.xml
>    [junit4]   2> 1079248 INFO
> (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [
> ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig-caching.xml
>    [junit4]   2> 1079250 INFO
> (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [
> ] o.a.s.c.c.SolrZkClient makePath:
> /configs/conf1/conditional.updateprocessor.js
>    [junit4]   2> 1079251 INFO
> (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [
> ] o.a.s.c.c.SolrZkClient makePath:
> /configs/conf1/solrconfig-indexconfig-mergepolicyfactory.xml
>    [junit4]   2> 1079252 INFO
> (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [
> ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/compoundDictionary.txt
>    [junit4]   2> 1079253 INFO
> (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [
> ] o.a.s.c.c.SolrZkClient makePath:
> /configs/conf1/solrconfig-tieredmergepolicyfactory.xml
>    [junit4]   2> 1079254 INFO
> (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [
> ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/phrasesuggest.txt
>    [junit4]   2> 1079255 INFO
> (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [
> ] o.a.s.c.c.SolrZkClient makePath:
> /configs/conf1/solrconfig-transformers.xml
>    [junit4]   2> 1079256 INFO
> (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [
> ] o.a.s.c.c.SolrZkClient makePath:
> /configs/conf1/bad-schema-not-indexed-but-norms.xml
>    [junit4]   2> 1079257 INFO
> (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [
> ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/blendedInfixSuggest.txt
>    [junit4]   2> 1079258 INFO
> (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [
> ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema-phrasesuggest.xml
>    [junit4]   2> 1079259 INFO
> (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [
> ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/hunspell-test.dic
>    [junit4]   2> 1079260 INFO
> (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [
> ] o.a.s.c.c.SolrZkClient makePath:
> /configs/conf1/schema-HighlighterMaxOffsetTest.xml
>    [junit4]   2> 1079261 INFO
> (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [
> ] o.a.s.c.c.SolrZkClient makePath:
> /configs/conf1/schema-rest-lucene-match-version.xml
>    [junit4]   2> 1079262 INFO
> (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [
> ] o.a.s.c.c.SolrZkClient makePath:
> /configs/conf1/solrconfig-functionquery.xml
>    [junit4]   2> 1079264 INFO
> (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [
> ] o.a.s.c.c.SolrZkClient makePath:
> /configs/conf1/solrconfig-externalversionconstraint.xml
>    [junit4]   2> 1079265 INFO
> (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [
> ] o.a.s.c.c.SolrZkClient makePath:
> /configs/conf1/bad-schema-codec-global-vs-ft-mismatch.xml
>    [junit4]   2> 1079266 INFO
> (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [
> ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema-sim.xml
>    [junit4]   2> 1079267 INFO
> (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [
> ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema-docValuesMulti.xml
>    [junit4]   2> 1079268 INFO
> (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [
> ] o.a.s.c.c.SolrZkClient makePath:
> /configs/conf1/solrconfig-logmergepolicyfactory.xml
>    [junit4]   2> 1079269 INFO
> (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [
> ] o.a.s.c.c.SolrZkClient makePath:
> /configs/conf1/solrconfig-collapseqparser.xml
>    [junit4]   2> 1079270 INFO
> (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [
> ] o.a.s.c.c.SolrZkClient makePath:
> /configs/conf1/solrconfig-analytics-query.xml
>    [junit4]   2> 1079271 INFO
> (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [
> ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema-sweetspot.xml
>    [junit4]   2> 1079273 INFO
> (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [
> ] o.a.s.c.c.SolrZkClient makePath:
> /configs/conf1/solrconfig-searcher-listeners1.xml
>    [junit4]   2> 1079274 INFO
> (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [
> ] o.a.s.c.c.SolrZkClient makePath:
> /configs/conf1/schema-id-and-version-fields-only.xml
>    [junit4]   2> 1079275 INFO
> (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [
> ] o.a.s.c.c.SolrZkClient makePath:
> /configs/conf1/solrconfig-classification.xml
>    [junit4]   2> 1079276 INFO
> (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [
> ] o.a.s.c.c.SolrZkClient makePath:
> /configs/conf1/bad-solrconfig-schema-mutable-but-not-managed.xml
>    [junit4]   2> 1079277 INFO
> (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [
> ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/hunspell-test.aff
>    [junit4]   2> 1079278 INFO
> (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [
> ] o.a.s.c.c.SolrZkClient makePath:
> /configs/conf1/solrconfig-response-log-component.xml
>    [junit4]   2> 1079279 INFO
> (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [
> ] o.a.s.c.c.SolrZkClient makePath:
> /configs/conf1/bad-schema-sim-default-does-not-exist.xml
>    [junit4]   2> 1079280 INFO
> (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [
> ] o.a.s.c.c.SolrZkClient makePath:
> /configs/conf1/bad-schema-dynamicfield-required.xml
>    [junit4]   2> 1079282 INFO
> (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [
> ] o.a.s.c.c.SolrZkClient makePath:
> /configs/conf1/bad-schema-not-indexed-but-pos.xml
>    [junit4]   2> 1079283 INFO
> (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [
> ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig-hash.xml
>    [junit4]   2> 1079284 INFO
> (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [
> ] o.a.s.c.c.SolrZkClient makePath:
> /configs/conf1/solrconfig-components-name.xml
>    [junit4]   2> 1079285 INFO
> (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [
> ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/bad-mp-solrconfig.xml
>    [junit4]   2> 1079286 INFO
> (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [
> ] o.a.s.c.c.SolrZkClient makePath:
> /configs/conf1/solrconfig.snippet.randomindexconfig.xml
>    [junit4]   2> 1079287 INFO
> (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [
>
> [...truncated too long message...]
>
> 0.0-M20.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/bcprov-jdk15-1.45.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/cglib-nodep-2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/commons-collections-3.2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/commons-math3-3.4.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/easymock-3.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/ehcache-core-2.4.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/hadoop-common-2.7.2-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/hadoop-hdfs-2.7.2-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/hadoop-minikdc-2.7.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/htrace-core-3.2.0-incubating.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jackson-annotations-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jackson-core-asl-1.9.13.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jackson-databind-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jackson-mapper-asl-1.9.13.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jersey-core-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jersey-server-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jetty-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jetty-util-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/mina-core-2.0.0-M5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/mockito-core-1.9.5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/netty-all-4.0.36.Final.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/objenesis-1.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/presto-parser-0.122.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/slice-0.10.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/analysis/icu/lucene-analyzers-icu-6.2.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/contrib/solr-analysis-extras/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/queryparser/classes/test:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/contrib/analysis-extras/lib/icu4j-56.1.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-launcher.jar:/x1/jenkins/.ant/lib/ivy-2.3.0.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-antlr.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-regexp.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-resolver.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-net.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jmf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-oro.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-xalan2.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-logging.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jsch.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit4.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-javamail.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jai.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jdepend.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bsf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-swing.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-netrexx.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bcel.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-testutil.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-log4j.jar:/x1/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.8/lib/tools.jar:/x1/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.3.4.jar
> com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/temp/junit4-J0-20160721_144740_155.events
> @/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/temp/junit4-J0-20160721_144740_155.suites
> -stdin
>    [junit4] ERROR: JVM J0 ended with an exception: Quit event not received
> from the forked process? This may indicate JVM crash or runner bugs.
>    [junit4]     at
> com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1501)
>    [junit4]     at
> com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:121)
>    [junit4]     at
> com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:955)
>    [junit4]     at
> com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:952)
>    [junit4]     at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>    [junit4]     at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>    [junit4]     at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>    [junit4]     at java.lang.Thread.run(Thread.java:745)
>
> BUILD FAILED
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:771:
> The following error occurred while executing this line:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:707:
> The following error occurred while executing this line:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:59:
> The following error occurred while executing this line:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build.xml:233:
> The following error occurred while executing this line:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/common-build.xml:536:
> The following error occurred while executing this line:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/common-build.xml:1443:
> The following error occurred while executing this line:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/common-build.xml:984:
> At least one slave process threw an exception, first: Quit event not
> received from the forked process? This may indicate JVM crash or runner
> bugs.
>
> Total time: 334 minutes 53 seconds
> Build step 'Invoke Ant' marked build as failure
> Archiving artifacts
> ERROR: Failed to archive {README.txt=README.txt,
> heapdumps/java_pid27116.hprof=heapdumps/java_pid27116.hprof,
> solr/build/solr-core/test/temp/junit4-J1-20160721_144740_155.events=solr/build/solr-core/test/temp/junit4-J1-20160721_144740_155.events,
> solr/build/solr-core/test/temp/junit4-J2-20160721_144740_160.events=solr/build/solr-core/test/temp/junit4-J2-20160721_144740_160.events,
> heapdumps/java_pid3660.hprof=heapdumps/java_pid3660.hprof,
> solr/build/solr-core/test/temp/junit4-J0-20160721_144740_155.events=solr/build/solr-core/test/temp/junit4-J0-20160721_144740_155.events}
> due to internal error; falling back to full archiving
> java.lang.NullPointerException
>         at jsync.protocol.ChangeInputStream.read(ChangeInputStream.java:69)
>         at
> com.cloudbees.jenkins.plugins.jsync.archiver.MD5DigestingInputStream.read(MD5DigestingInputStream.java:39)
>         at
> com.google.common.io.LimitInputStream.read(LimitInputStream.java:79)
>         at java.io.FilterInputStream.read(FilterInputStream.java:107)
>         at com.google.common.io.ByteStreams.copy(ByteStreams.java:193)
>         at
> jsync.protocol.FileSequenceReader.read(FileSequenceReader.java:35)
>         at
> com.cloudbees.jenkins.plugins.jsync.archiver.JSyncArtifactManager.remoteSync(JSyncArtifactManager.java:148)
>         at
> com.cloudbees.jenkins.plugins.jsync.archiver.JSyncArtifactManager.archive(JSyncArtifactManager.java:72)
>         at hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:236)
>         at
> hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:78)
>         at
> hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
>         at
> hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:782)
>         at
> hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:723)
>         at hudson.model.Build$BuildExecution.post2(Build.java:185)
>         at
> hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:668)
>         at hudson.model.Run.execute(Run.java:1763)
>         at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
>         at
> hudson.model.ResourceController.execute(ResourceController.java:98)
>         at hudson.model.Executor.run(Executor.java:410)
> Recording test results
> Email was triggered for: Failure - Any
> Sending email for trigger: Failure - Any
>
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>

[JENKINS] Lucene-Solr-NightlyTests-6.x - Build # 125 - Failure

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-6.x/125/

6 tests failed.
FAILED:  org.apache.lucene.index.TestDuelingCodecsAtNight.testBigEquals

Error Message:
Java heap space

Stack Trace:
java.lang.OutOfMemoryError: Java heap space
	at __randomizedtesting.SeedInfo.seed([62829676FE994519:8186CDF35582815E]:0)
	at org.apache.lucene.util.fst.BytesStore.writeBytes(BytesStore.java:112)
	at org.apache.lucene.util.fst.ByteSequenceOutputs.write(ByteSequenceOutputs.java:120)
	at org.apache.lucene.util.fst.ByteSequenceOutputs.write(ByteSequenceOutputs.java:35)
	at org.apache.lucene.util.fst.Outputs.writeFinalOutput(Outputs.java:60)
	at org.apache.lucene.util.fst.FST.pack(FST.java:1714)
	at org.apache.lucene.util.fst.Builder.finish(Builder.java:500)
	at org.apache.lucene.codecs.memory.MemoryPostingsFormat$TermsWriter.finish(MemoryPostingsFormat.java:267)
	at org.apache.lucene.codecs.memory.MemoryPostingsFormat$MemoryFieldsConsumer.write(MemoryPostingsFormat.java:401)
	at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsWriter.write(PerFieldPostingsFormat.java:198)
	at org.apache.lucene.codecs.FieldsConsumer.merge(FieldsConsumer.java:105)
	at org.apache.lucene.index.SegmentMerger.mergeTerms(SegmentMerger.java:216)
	at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:101)
	at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4312)
	at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3889)
	at org.apache.lucene.index.SerialMergeScheduler.merge(SerialMergeScheduler.java:40)
	at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:2052)
	at org.apache.lucene.index.IndexWriter.doAfterSegmentFlushed(IndexWriter.java:4953)
	at org.apache.lucene.index.DocumentsWriter$MergePendingEvent.process(DocumentsWriter.java:742)
	at org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:4991)
	at org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:4982)
	at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1565)
	at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1307)
	at org.apache.lucene.index.RandomIndexWriter.addDocument(RandomIndexWriter.java:171)
	at org.apache.lucene.index.TestDuelingCodecs.createRandomIndex(TestDuelingCodecs.java:139)
	at org.apache.lucene.index.TestDuelingCodecsAtNight.testBigEquals(TestDuelingCodecsAtNight.java:34)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)


FAILED:  org.apache.lucene.search.TestDoubleRangeFieldQueries.testMultiValued

Error Message:
wrong hit (first of possibly more):  FAIL: id=204 should not match but did  queryBox=Box(Infinity TO Infinity)  box=Box(Infinity TO Infinity)  queryType=CONTAINS  deleted?=false

Stack Trace:
java.lang.AssertionError: wrong hit (first of possibly more):

FAIL: id=204 should not match but did
 queryBox=Box(Infinity TO Infinity)
 box=Box(Infinity TO Infinity)
 queryType=CONTAINS
 deleted?=false
	at __randomizedtesting.SeedInfo.seed([AE24BFDDECD216AC:7A04DBEF221056E4]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.lucene.search.BaseRangeFieldQueryTestCase.verify(BaseRangeFieldQueryTestCase.java:278)
	at org.apache.lucene.search.BaseRangeFieldQueryTestCase.doTestRandom(BaseRangeFieldQueryTestCase.java:154)
	at org.apache.lucene.search.BaseRangeFieldQueryTestCase.testMultiValued(BaseRangeFieldQueryTestCase.java:73)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.lucene.search.TestDoubleRangeFieldQueries.testRandomBig

Error Message:
wrong hit (first of possibly more):  FAIL: id=49 should not match but did  queryBox=Box(Infinity TO Infinity)  box=Box(Infinity TO Infinity)  queryType=CONTAINS  deleted?=false

Stack Trace:
java.lang.AssertionError: wrong hit (first of possibly more):

FAIL: id=49 should not match but did
 queryBox=Box(Infinity TO Infinity)
 box=Box(Infinity TO Infinity)
 queryType=CONTAINS
 deleted?=false
	at __randomizedtesting.SeedInfo.seed([AE24BFDDECD216AC:2973C2527D8B6A2C]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.lucene.search.BaseRangeFieldQueryTestCase.verify(BaseRangeFieldQueryTestCase.java:278)
	at org.apache.lucene.search.BaseRangeFieldQueryTestCase.doTestRandom(BaseRangeFieldQueryTestCase.java:154)
	at org.apache.lucene.search.BaseRangeFieldQueryTestCase.testRandomBig(BaseRangeFieldQueryTestCase.java:69)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.search.stats.TestDistribIDF.testMultiCollectionQuery

Error Message:
Doc1 score=0.9808292 Doc2 score=0.9808292 expected:<1> but was:<0>

Stack Trace:
java.lang.AssertionError: Doc1 score=0.9808292 Doc2 score=0.9808292 expected:<1> but was:<0>
	at __randomizedtesting.SeedInfo.seed([1F45A3F87F67AC9C:E3664C9A311A9E0]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.failNotEquals(Assert.java:647)
	at org.junit.Assert.assertEquals(Assert.java:128)
	at org.junit.Assert.assertEquals(Assert.java:472)
	at org.apache.solr.search.stats.TestDistribIDF.testMultiCollectionQuery(TestDistribIDF.java:183)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  junit.framework.TestSuite.org.apache.solr.handler.TestReplicationHandler

Error Message:
ObjectTracker found 2 object(s) that were not released!!! [NRTCachingDirectory, NRTCachingDirectory]

Stack Trace:
java.lang.AssertionError: ObjectTracker found 2 object(s) that were not released!!! [NRTCachingDirectory, NRTCachingDirectory]
	at __randomizedtesting.SeedInfo.seed([1F45A3F87F67AC9C]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.junit.Assert.assertNull(Assert.java:551)
	at org.apache.solr.SolrTestCaseJ4.teardownTestCases(SolrTestCaseJ4.java:258)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:834)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Timeout occured while waiting response from server at: http://127.0.0.1:36062

Stack Trace:
org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:36062
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:601)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:259)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:248)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.makeRequest(CollectionsAPIDistributedZkTest.java:399)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testErrorHandling(CollectionsAPIDistributedZkTest.java:515)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test(CollectionsAPIDistributedZkTest.java:179)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.net.SocketTimeoutException: Read timed out
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
	at java.net.SocketInputStream.read(SocketInputStream.java:170)
	at java.net.SocketInputStream.read(SocketInputStream.java:141)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:160)
	at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:84)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:273)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
	at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
	at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)
	at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)
	at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197)
	at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)
	at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)
	at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685)
	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487)
	at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:495)
	... 47 more




Build Log:
[...truncated 1725 lines...]
   [junit4] Suite: org.apache.lucene.index.TestDuelingCodecsAtNight
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestDuelingCodecsAtNight -Dtests.method=testBigEquals -Dtests.seed=62829676FE994519 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=ar-AE -Dtests.timezone=America/Managua -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
   [junit4] ERROR   2423s J1 | TestDuelingCodecsAtNight.testBigEquals <<<
   [junit4]    > Throwable #1: java.lang.OutOfMemoryError: Java heap space
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([62829676FE994519:8186CDF35582815E]:0)
   [junit4]    > 	at org.apache.lucene.util.fst.BytesStore.writeBytes(BytesStore.java:112)
   [junit4]    > 	at org.apache.lucene.util.fst.ByteSequenceOutputs.write(ByteSequenceOutputs.java:120)
   [junit4]    > 	at org.apache.lucene.util.fst.ByteSequenceOutputs.write(ByteSequenceOutputs.java:35)
   [junit4]    > 	at org.apache.lucene.util.fst.Outputs.writeFinalOutput(Outputs.java:60)
   [junit4]    > 	at org.apache.lucene.util.fst.FST.pack(FST.java:1714)
   [junit4]    > 	at org.apache.lucene.util.fst.Builder.finish(Builder.java:500)
   [junit4]    > 	at org.apache.lucene.codecs.memory.MemoryPostingsFormat$TermsWriter.finish(MemoryPostingsFormat.java:267)
   [junit4]    > 	at org.apache.lucene.codecs.memory.MemoryPostingsFormat$MemoryFieldsConsumer.write(MemoryPostingsFormat.java:401)
   [junit4]    > 	at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsWriter.write(PerFieldPostingsFormat.java:198)
   [junit4]    > 	at org.apache.lucene.codecs.FieldsConsumer.merge(FieldsConsumer.java:105)
   [junit4]    > 	at org.apache.lucene.index.SegmentMerger.mergeTerms(SegmentMerger.java:216)
   [junit4]    > 	at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:101)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4312)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3889)
   [junit4]    > 	at org.apache.lucene.index.SerialMergeScheduler.merge(SerialMergeScheduler.java:40)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:2052)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.doAfterSegmentFlushed(IndexWriter.java:4953)
   [junit4]    > 	at org.apache.lucene.index.DocumentsWriter$MergePendingEvent.process(DocumentsWriter.java:742)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:4991)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:4982)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1565)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1307)
   [junit4]    > 	at org.apache.lucene.index.RandomIndexWriter.addDocument(RandomIndexWriter.java:171)
   [junit4]    > 	at org.apache.lucene.index.TestDuelingCodecs.createRandomIndex(TestDuelingCodecs.java:139)
   [junit4]    > 	at org.apache.lucene.index.TestDuelingCodecsAtNight.testBigEquals(TestDuelingCodecsAtNight.java:34)
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/test/J1/temp/lucene.index.TestDuelingCodecsAtNight_62829676FE994519-001
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene62): {}, docValues:{}, maxPointsInLeafNode=1581, maxMBSortInHeap=6.757363644236303, sim=RandomSimilarity(queryNorm=true,coord=yes): {titleTokenized=DFI(Standardized), body=DFR I(n)L1}, locale=ar-AE, timezone=America/Managua
   [junit4]   2> NOTE: Linux 3.13.0-85-generic amd64/Oracle Corporation 1.8.0_74 (64-bit)/cpus=4,threads=1,free=269638936,total=405274624
   [junit4]   2> NOTE: All tests run in this JVM: [TestTermdocPerf, TestLucene60PointsFormat, TestInfoStream, TestPackedInts, TestTransactions, TestBooleanRewrites, TestAttributeSource, TestFilterIterator, TestDocIDMerger, TestRegExp, TestPolygon2D, TestParallelLeafReader, TestAddIndexes, TestSpanExplanations, TestPositiveScoresOnlyCollector, TestDirectPacked, TestSetOnce, TestVirtualMethod, TestSloppyPhraseQuery2, TestFixedBitSet, TestTimSorter, TestVersion, TestAllFilesHaveChecksumFooter, TestCharsRef, TestDoc, TestAssertions, TestSleepingLockWrapper, TestSPIClassIterator, TestBufferedChecksum, TestRegexpRandom, TestHighCompressionMode, TestRegexpQuery, TestIndexCommit, TestReadOnlyIndex, TestTermVectorsWriter, TestParallelTermEnum, TestFieldMaskingSpanQuery, TestPerSegmentDeletes, TestNewestSegment, TestScorerPerf, TestTwoPhaseCommitTool, TestGeoEncodingUtils, TestIndexWriterExceptions, TestSearchForDuplicates, TestSimpleSearchEquivalence, TestTermVectors, TestDocument, TestFileSwitchDirectory, TestBasics, TestIndexSorting, TestPositionIncrement, TestApproximationSearchEquivalence, TestIndexSearcher, Test2BNumericDocValues, TestBinaryDocument, TestLockFactory, TestDocIdsWriter, TestPerFieldPostingsFormat, TestSnapshotDeletionPolicy, TestSortRescorer, TestPostingsOffsets, TestSegmentReader, MultiCollectorTest, TestStopFilter, TestLucene50TermVectorsFormat, TestSpans, TestNIOFSDirectory, TestFieldReuse, TestSparseFixedBitSet, TestDocumentsWriterDeleteQueue, TestBinaryTerms, TestFilterSpans, TestThreadedForceMerge, TestPerFieldDocValuesFormat, TestIndexWriterReader, TestIndexWriterMerging, TestFSTs, TestIndexWriterWithThreads, TestMultiMMap, TestDeletionPolicy, TestFlushByRamOrCountsPolicy, TestIndexWriterMergePolicy, TestDuelingCodecs, TestBoolean2, TestAtomicUpdate, TestStressAdvance, TestNorms, TestIndexWriterCommit, TestDirectoryReaderReopen, TestTimeLimitingCollector, TestSearchWithThreads, TestSearchAfter, TestDocumentsWriterStallControl, TestTopDocsMerge, TestIndexWriterUnicode, TestNRTReaderWithThreads, TestTieredMergePolicy, TestLevenshteinAutomata, TestByteSlices, TestTermsEnum, TestToken, TestCustomSearcherSort, TestHugeRamFile, TestLegacyNumericUtils, TestTransactionRollback, TestFlex, TestUTF32ToUTF8, TestTermRangeQuery, TestMultiTermConstantScore, TestFuzzyQuery, TestExternalCodecs, TestPrefixInBooleanQuery, TestComplexExplanationsOfNonMatches, TestUnicodeUtil, TestPrefixCodedTerms, TestSumDocFreq, TestConstantScoreQuery, TestElevationComparator, TestIndexWriterConfig, TestMultiTermQueryRewrites, TestSimilarity, TestRollingBuffer, TestFilterLeafReader, TestCachingCollector, TestRamUsageEstimator, TestNoMergeScheduler, TestPhrasePrefixQuery, TestCharFilter, TestCloseableThreadLocal, TestScoreCachingWrappingScorer, TestIntsRef, TestIOUtils, Test2BPagedBytes, TestCharArraySet, TestWordlistLoader, TestPackedTokenAttributeImpl, TestBlockPostingsFormat, TestGeoUtils, TestDocInverterPerFieldErrorInfo, TestDuelingCodecsAtNight]
   [junit4] Completed [435/435 (1!)] on J1 in 2466.59s, 3 tests, 1 error <<< FAILURES!

[...truncated 1 lines...]
   [junit4] JVM J1: stdout was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/test/temp/junit4-J1-20160721_111954_897.sysout
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] java.lang.OutOfMemoryError: Java heap space
   [junit4] Dumping heap to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/heapdumps/java_pid27116.hprof ...
   [junit4] Heap dump file created [422506181 bytes in 0.847 secs]
   [junit4] <<< JVM J1: EOF ----

[...truncated 7226 lines...]
   [junit4] Suite: org.apache.lucene.search.TestDoubleRangeFieldQueries
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestDoubleRangeFieldQueries -Dtests.method=testMultiValued -Dtests.seed=AE24BFDDECD216AC -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=hu -Dtests.timezone=Europe/San_Marino -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
   [junit4] FAILURE 4.20s J0 | TestDoubleRangeFieldQueries.testMultiValued <<<
   [junit4]    > Throwable #1: java.lang.AssertionError: wrong hit (first of possibly more):
   [junit4]    > FAIL: id=204 should not match but did
   [junit4]    >  queryBox=Box(Infinity TO Infinity)
   [junit4]    >  box=Box(Infinity TO Infinity)
   [junit4]    >  queryType=CONTAINS
   [junit4]    >  deleted?=false
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([AE24BFDDECD216AC:7A04DBEF221056E4]:0)
   [junit4]    > 	at org.apache.lucene.search.BaseRangeFieldQueryTestCase.verify(BaseRangeFieldQueryTestCase.java:278)
   [junit4]    > 	at org.apache.lucene.search.BaseRangeFieldQueryTestCase.doTestRandom(BaseRangeFieldQueryTestCase.java:154)
   [junit4]    > 	at org.apache.lucene.search.BaseRangeFieldQueryTestCase.testMultiValued(BaseRangeFieldQueryTestCase.java:73)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestDoubleRangeFieldQueries -Dtests.method=testRandomBig -Dtests.seed=AE24BFDDECD216AC -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=hu -Dtests.timezone=Europe/San_Marino -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
   [junit4] FAILURE 35.1s J0 | TestDoubleRangeFieldQueries.testRandomBig <<<
   [junit4]    > Throwable #1: java.lang.AssertionError: wrong hit (first of possibly more):
   [junit4]    > FAIL: id=49 should not match but did
   [junit4]    >  queryBox=Box(Infinity TO Infinity)
   [junit4]    >  box=Box(Infinity TO Infinity)
   [junit4]    >  queryType=CONTAINS
   [junit4]    >  deleted?=false
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([AE24BFDDECD216AC:2973C2527D8B6A2C]:0)
   [junit4]    > 	at org.apache.lucene.search.BaseRangeFieldQueryTestCase.verify(BaseRangeFieldQueryTestCase.java:278)
   [junit4]    > 	at org.apache.lucene.search.BaseRangeFieldQueryTestCase.doTestRandom(BaseRangeFieldQueryTestCase.java:154)
   [junit4]    > 	at org.apache.lucene.search.BaseRangeFieldQueryTestCase.testRandomBig(BaseRangeFieldQueryTestCase.java:69)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/sandbox/test/J0/temp/lucene.search.TestDoubleRangeFieldQueries_AE24BFDDECD216AC-001
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene62): {id=PostingsFormat(name=Memory doPackFST= false)}, docValues:{id=DocValuesFormat(name=Memory)}, maxPointsInLeafNode=1844, maxMBSortInHeap=6.78445457997637, sim=ClassicSimilarity, locale=hu, timezone=Europe/San_Marino
   [junit4]   2> NOTE: Linux 3.13.0-85-generic amd64/Oracle Corporation 1.8.0_74 (64-bit)/cpus=4,threads=1,free=179129064,total=341311488
   [junit4]   2> NOTE: All tests run in this JVM: [FuzzyLikeThisQueryTest, TestDocValuesTermsQuery, TestTermAutomatonQuery, TestLatLonPoint, TestBigIntegerPoint, TestDoubleRangeField, TestHalfFloatPoint, TestInetAddressPoint, TestLatLonDocValuesField, TestPayloadSpanUtil, TestDoubleRangeFieldQueries]
   [junit4] Completed [18/20 (1!)] on J0 in 42.60s, 5 tests, 2 failures <<< FAILURES!

[...truncated 2751 lines...]
   [junit4] Suite: org.apache.solr.search.stats.TestDistribIDF
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.search.stats.TestDistribIDF_1F45A3F87F67AC9C-001/init-core-data-001
   [junit4]   2> 1078911 INFO  (SUITE-TestDistribIDF-seed#[1F45A3F87F67AC9C]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
   [junit4]   2> 1078915 INFO  (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testMultiCollectionQuery
   [junit4]   2> 1078915 INFO  (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 1078917 INFO  (Thread-1456) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1078917 INFO  (Thread-1456) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 1079017 INFO  (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [    ] o.a.s.c.ZkTestServer start zk server on port:52388
   [junit4]   2> 1079017 INFO  (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1079018 INFO  (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1079025 INFO  (zkCallback-2741-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@64cee6ba name:ZooKeeperConnection Watcher:127.0.0.1:52388 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1079026 INFO  (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1079026 INFO  (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1079026 INFO  (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr/solr.xml
   [junit4]   2> 1079041 INFO  (jetty-launcher-2740-thread-1) [    ] o.e.j.s.Server jetty-9.3.8.v20160314
   [junit4]   2> 1079041 INFO  (jetty-launcher-2740-thread-2) [    ] o.e.j.s.Server jetty-9.3.8.v20160314
   [junit4]   2> 1079042 INFO  (jetty-launcher-2740-thread-3) [    ] o.e.j.s.Server jetty-9.3.8.v20160314
   [junit4]   2> 1079043 INFO  (jetty-launcher-2740-thread-1) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@52f75917{/solr,null,AVAILABLE}
   [junit4]   2> 1079045 INFO  (jetty-launcher-2740-thread-1) [    ] o.e.j.s.ServerConnector Started ServerConnector@216f0751{HTTP/1.1,[http/1.1]}{127.0.0.1:54619}
   [junit4]   2> 1079045 INFO  (jetty-launcher-2740-thread-1) [    ] o.e.j.s.Server Started @1083810ms
   [junit4]   2> 1079045 INFO  (jetty-launcher-2740-thread-1) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=54619}
   [junit4]   2> 1079045 INFO  (jetty-launcher-2740-thread-1) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@73d16e93
   [junit4]   2> 1079045 INFO  (jetty-launcher-2740-thread-1) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.search.stats.TestDistribIDF_1F45A3F87F67AC9C-001/tempDir-001/node1'
   [junit4]   2> 1079045 INFO  (jetty-launcher-2740-thread-1) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1079045 INFO  (jetty-launcher-2740-thread-1) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1079045 INFO  (jetty-launcher-2740-thread-1) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1079047 INFO  (jetty-launcher-2740-thread-1) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1079049 INFO  (jetty-launcher-2740-thread-2) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5e041f38{/solr,null,AVAILABLE}
   [junit4]   2> 1079049 INFO  (jetty-launcher-2740-thread-2) [    ] o.e.j.s.ServerConnector Started ServerConnector@347ecf61{HTTP/1.1,[http/1.1]}{127.0.0.1:49958}
   [junit4]   2> 1079049 INFO  (jetty-launcher-2740-thread-2) [    ] o.e.j.s.Server Started @1083815ms
   [junit4]   2> 1079049 INFO  (jetty-launcher-2740-thread-2) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=49958}
   [junit4]   2> 1079049 INFO  (jetty-launcher-2740-thread-2) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@73d16e93
   [junit4]   2> 1079049 INFO  (jetty-launcher-2740-thread-2) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.search.stats.TestDistribIDF_1F45A3F87F67AC9C-001/tempDir-001/node2'
   [junit4]   2> 1079049 INFO  (jetty-launcher-2740-thread-2) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1079049 INFO  (jetty-launcher-2740-thread-2) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1079050 INFO  (jetty-launcher-2740-thread-2) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1079051 INFO  (jetty-launcher-2740-thread-2) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1079052 INFO  (jetty-launcher-2740-thread-3) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@31a6e920{/solr,null,AVAILABLE}
   [junit4]   2> 1079052 INFO  (jetty-launcher-2740-thread-3) [    ] o.e.j.s.ServerConnector Started ServerConnector@7d06c7f9{HTTP/1.1,[http/1.1]}{127.0.0.1:46930}
   [junit4]   2> 1079052 INFO  (jetty-launcher-2740-thread-3) [    ] o.e.j.s.Server Started @1083818ms
   [junit4]   2> 1079052 INFO  (jetty-launcher-2740-thread-3) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=46930}
   [junit4]   2> 1079053 INFO  (jetty-launcher-2740-thread-3) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@73d16e93
   [junit4]   2> 1079053 INFO  (jetty-launcher-2740-thread-3) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.search.stats.TestDistribIDF_1F45A3F87F67AC9C-001/tempDir-001/node3'
   [junit4]   2> 1079053 INFO  (jetty-launcher-2740-thread-3) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1079053 INFO  (jetty-launcher-2740-thread-3) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1079053 INFO  (jetty-launcher-2740-thread-3) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1079067 INFO  (jetty-launcher-2740-thread-3) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1079071 INFO  (zkCallback-2742-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6e1c60a2 name:ZooKeeperConnection Watcher:127.0.0.1:52388/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1079071 INFO  (jetty-launcher-2740-thread-1) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1079072 INFO  (jetty-launcher-2740-thread-1) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1079072 INFO  (jetty-launcher-2740-thread-1) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 1079091 INFO  (zkCallback-2743-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@ecde4e4 name:ZooKeeperConnection Watcher:127.0.0.1:52388/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1079092 INFO  (jetty-launcher-2740-thread-1) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.search.stats.TestDistribIDF_1F45A3F87F67AC9C-001/tempDir-001/node1/.
   [junit4]   2> 1079092 INFO  (jetty-launcher-2740-thread-2) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1079092 INFO  (jetty-launcher-2740-thread-1) [    ] o.a.s.c.CoreContainer New CoreContainer 1005926444
   [junit4]   2> 1079092 INFO  (jetty-launcher-2740-thread-1) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.search.stats.TestDistribIDF_1F45A3F87F67AC9C-001/tempDir-001/node1]
   [junit4]   2> 1079092 INFO  (jetty-launcher-2740-thread-2) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1079092 WARN  (jetty-launcher-2740-thread-1) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.search.stats.TestDistribIDF_1F45A3F87F67AC9C-001/tempDir-001/node1/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.search.stats.TestDistribIDF_1F45A3F87F67AC9C-001/tempDir-001/node1/lib
   [junit4]   2> 1079092 INFO  (jetty-launcher-2740-thread-1) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,connectionsEvictorSleepDelay : 5000,maxConnectionIdleTime : 40000,
   [junit4]   2> 1079093 INFO  (jetty-launcher-2740-thread-1) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1079093 INFO  (jetty-launcher-2740-thread-2) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 1079092 INFO  (zkCallback-2744-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@158ebf96 name:ZooKeeperConnection Watcher:127.0.0.1:52388/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1079094 INFO  (jetty-launcher-2740-thread-1) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1079094 INFO  (jetty-launcher-2740-thread-3) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1079094 INFO  (jetty-launcher-2740-thread-1) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1079094 INFO  (jetty-launcher-2740-thread-1) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:52388/solr
   [junit4]   2> 1079094 INFO  (jetty-launcher-2740-thread-3) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1079094 INFO  (jetty-launcher-2740-thread-1) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1079094 INFO  (jetty-launcher-2740-thread-1) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1079095 INFO  (jetty-launcher-2740-thread-3) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 1079096 INFO  (jetty-launcher-2740-thread-1) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1079096 INFO  (zkCallback-2747-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7d1f4e95 name:ZooKeeperConnection Watcher:127.0.0.1:52388 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1079096 INFO  (jetty-launcher-2740-thread-1) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1079096 INFO  (jetty-launcher-2740-thread-1) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1079097 INFO  (jetty-launcher-2740-thread-1) [n:127.0.0.1:54619_solr    ] o.a.s.c.ZkController Added new OnReconnect listener org.apache.solr.cloud.ZkController$$Lambda$73/369477848@5c1b8b00
   [junit4]   2> 1079098 INFO  (jetty-launcher-2740-thread-1) [n:127.0.0.1:54619_solr    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1079099 INFO  (zkCallback-2748-thread-1-processing-n:127.0.0.1:54619_solr) [n:127.0.0.1:54619_solr    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@70735b96 name:ZooKeeperConnection Watcher:127.0.0.1:52388/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1079099 INFO  (jetty-launcher-2740-thread-1) [n:127.0.0.1:54619_solr    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1079100 INFO  (jetty-launcher-2740-thread-1) [n:127.0.0.1:54619_solr    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 1079101 INFO  (jetty-launcher-2740-thread-2) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.search.stats.TestDistribIDF_1F45A3F87F67AC9C-001/tempDir-001/node2/.
   [junit4]   2> 1079101 INFO  (jetty-launcher-2740-thread-2) [    ] o.a.s.c.CoreContainer New CoreContainer 1804464227
   [junit4]   2> 1079101 INFO  (jetty-launcher-2740-thread-2) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.search.stats.TestDistribIDF_1F45A3F87F67AC9C-001/tempDir-001/node2]
   [junit4]   2> 1079101 WARN  (jetty-launcher-2740-thread-2) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.search.stats.TestDistribIDF_1F45A3F87F67AC9C-001/tempDir-001/node2/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.search.stats.TestDistribIDF_1F45A3F87F67AC9C-001/tempDir-001/node2/lib
   [junit4]   2> 1079101 INFO  (jetty-launcher-2740-thread-2) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,connectionsEvictorSleepDelay : 5000,maxConnectionIdleTime : 40000,
   [junit4]   2> 1079101 INFO  (jetty-launcher-2740-thread-1) [n:127.0.0.1:54619_solr    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 1079103 INFO  (jetty-launcher-2740-thread-1) [n:127.0.0.1:54619_solr    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 1079104 INFO  (jetty-launcher-2740-thread-1) [n:127.0.0.1:54619_solr    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 1079106 INFO  (jetty-launcher-2740-thread-1) [n:127.0.0.1:54619_solr    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 1079106 INFO  (jetty-launcher-2740-thread-2) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1079106 INFO  (jetty-launcher-2740-thread-3) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.search.stats.TestDistribIDF_1F45A3F87F67AC9C-001/tempDir-001/node3/.
   [junit4]   2> 1079106 INFO  (jetty-launcher-2740-thread-3) [    ] o.a.s.c.CoreContainer New CoreContainer 1499331252
   [junit4]   2> 1079106 INFO  (jetty-launcher-2740-thread-3) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.search.stats.TestDistribIDF_1F45A3F87F67AC9C-001/tempDir-001/node3]
   [junit4]   2> 1079106 WARN  (jetty-launcher-2740-thread-3) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.search.stats.TestDistribIDF_1F45A3F87F67AC9C-001/tempDir-001/node3/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.search.stats.TestDistribIDF_1F45A3F87F67AC9C-001/tempDir-001/node3/lib
   [junit4]   2> 1079108 INFO  (jetty-launcher-2740-thread-3) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,connectionsEvictorSleepDelay : 5000,maxConnectionIdleTime : 40000,
   [junit4]   2> 1079108 INFO  (jetty-launcher-2740-thread-3) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1079109 INFO  (jetty-launcher-2740-thread-3) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1079109 INFO  (jetty-launcher-2740-thread-3) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1079109 INFO  (jetty-launcher-2740-thread-3) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:52388/solr
   [junit4]   2> 1079109 INFO  (jetty-launcher-2740-thread-3) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1079109 INFO  (jetty-launcher-2740-thread-3) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1079109 INFO  (jetty-launcher-2740-thread-1) [n:127.0.0.1:54619_solr    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 1079106 INFO  (jetty-launcher-2740-thread-2) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1079110 INFO  (jetty-launcher-2740-thread-2) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1079110 INFO  (jetty-launcher-2740-thread-2) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:52388/solr
   [junit4]   2> 1079110 INFO  (jetty-launcher-2740-thread-2) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1079111 INFO  (jetty-launcher-2740-thread-2) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1079110 INFO  (jetty-launcher-2740-thread-3) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1079111 INFO  (jetty-launcher-2740-thread-2) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1079111 INFO  (zkCallback-2754-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@15b9783a name:ZooKeeperConnection Watcher:127.0.0.1:52388 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1079112 INFO  (jetty-launcher-2740-thread-3) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1079112 INFO  (jetty-launcher-2740-thread-3) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1079112 INFO  (zkCallback-2755-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@48ecfc7d name:ZooKeeperConnection Watcher:127.0.0.1:52388 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1079112 INFO  (jetty-launcher-2740-thread-1) [n:127.0.0.1:54619_solr    ] o.a.s.c.c.SolrZkClient makePath: /collections
   [junit4]   2> 1079113 INFO  (jetty-launcher-2740-thread-2) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1079113 INFO  (jetty-launcher-2740-thread-2) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1079113 INFO  (jetty-launcher-2740-thread-3) [n:127.0.0.1:46930_solr    ] o.a.s.c.ZkController Added new OnReconnect listener org.apache.solr.cloud.ZkController$$Lambda$73/369477848@2d43ea88
   [junit4]   2> 1079114 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x1560dfdace20007, likely client has closed socket
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 1079114 INFO  (jetty-launcher-2740-thread-1) [n:127.0.0.1:54619_solr    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 1079115 INFO  (jetty-launcher-2740-thread-3) [n:127.0.0.1:46930_solr    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1079115 INFO  (jetty-launcher-2740-thread-2) [n:127.0.0.1:49958_solr    ] o.a.s.c.ZkController Added new OnReconnect listener org.apache.solr.cloud.ZkController$$Lambda$73/369477848@4495a7b4
   [junit4]   2> 1079116 INFO  (jetty-launcher-2740-thread-1) [n:127.0.0.1:54619_solr    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 1079116 INFO  (jetty-launcher-2740-thread-2) [n:127.0.0.1:49958_solr    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1079117 INFO  (zkCallback-2756-thread-1-processing-n:127.0.0.1:46930_solr) [n:127.0.0.1:46930_solr    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5492d1b8 name:ZooKeeperConnection Watcher:127.0.0.1:52388/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1079117 INFO  (jetty-launcher-2740-thread-1) [n:127.0.0.1:54619_solr    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 1079117 INFO  (jetty-launcher-2740-thread-3) [n:127.0.0.1:46930_solr    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1079117 INFO  (zkCallback-2757-thread-1-processing-n:127.0.0.1:49958_solr) [n:127.0.0.1:49958_solr    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@efed6 name:ZooKeeperConnection Watcher:127.0.0.1:52388/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1079117 INFO  (jetty-launcher-2740-thread-2) [n:127.0.0.1:49958_solr    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1079118 INFO  (jetty-launcher-2740-thread-1) [n:127.0.0.1:54619_solr    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1079119 INFO  (jetty-launcher-2740-thread-1) [n:127.0.0.1:54619_solr    ] o.a.s.c.c.ZkStateReader Loaded empty cluster properties
   [junit4]   2> 1079120 INFO  (jetty-launcher-2740-thread-1) [n:127.0.0.1:54619_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (0)
   [junit4]   2> 1079123 INFO  (jetty-launcher-2740-thread-1) [n:127.0.0.1:54619_solr    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 1079123 INFO  (jetty-launcher-2740-thread-2) [n:127.0.0.1:49958_solr    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1079124 INFO  (jetty-launcher-2740-thread-2) [n:127.0.0.1:49958_solr    ] o.a.s.c.c.ZkStateReader Loaded empty cluster properties
   [junit4]   2> 1079124 INFO  (jetty-launcher-2740-thread-2) [n:127.0.0.1:49958_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (0)
   [junit4]   2> 1079124 INFO  (jetty-launcher-2740-thread-3) [n:127.0.0.1:46930_solr    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1079125 INFO  (jetty-launcher-2740-thread-1) [n:127.0.0.1:54619_solr    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 1079125 INFO  (jetty-launcher-2740-thread-3) [n:127.0.0.1:46930_solr    ] o.a.s.c.c.ZkStateReader Loaded empty cluster properties
   [junit4]   2> 1079125 INFO  (jetty-launcher-2740-thread-3) [n:127.0.0.1:46930_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (0)
   [junit4]   2> 1079126 INFO  (jetty-launcher-2740-thread-1) [n:127.0.0.1:54619_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1079126 INFO  (jetty-launcher-2740-thread-1) [n:127.0.0.1:54619_solr    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/96279825213423621-127.0.0.1:54619_solr-n_0000000000
   [junit4]   2> 1079127 INFO  (jetty-launcher-2740-thread-1) [n:127.0.0.1:54619_solr    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:54619_solr
   [junit4]   2> 1079127 INFO  (jetty-launcher-2740-thread-1) [n:127.0.0.1:54619_solr    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 1079128 INFO  (jetty-launcher-2740-thread-1) [n:127.0.0.1:54619_solr    ] o.a.s.c.Overseer Overseer (id=96279825213423621-127.0.0.1:54619_solr-n_0000000000) starting
   [junit4]   2> 1079128 INFO  (jetty-launcher-2740-thread-2) [n:127.0.0.1:49958_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1079128 INFO  (jetty-launcher-2740-thread-2) [n:127.0.0.1:49958_solr    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/96279825213423625-127.0.0.1:49958_solr-n_0000000001
   [junit4]   2> 1079129 INFO  (jetty-launcher-2740-thread-3) [n:127.0.0.1:46930_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1079129 INFO  (jetty-launcher-2740-thread-2) [n:127.0.0.1:49958_solr    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/96279825213423621-127.0.0.1:54619_solr-n_0000000000 to know if I could be the leader
   [junit4]   2> 1079129 INFO  (jetty-launcher-2740-thread-1) [n:127.0.0.1:54619_solr    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 1079130 INFO  (jetty-launcher-2740-thread-3) [n:127.0.0.1:46930_solr    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/96279825213423624-127.0.0.1:46930_solr-n_0000000002
   [junit4]   2> 1079130 INFO  (jetty-launcher-2740-thread-2) [n:127.0.0.1:49958_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:49958_solr
   [junit4]   2> 1079130 INFO  (jetty-launcher-2740-thread-2) [n:127.0.0.1:49958_solr    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:49958_solr
   [junit4]   2> 1079130 INFO  (jetty-launcher-2740-thread-3) [n:127.0.0.1:46930_solr    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/96279825213423625-127.0.0.1:49958_solr-n_0000000001 to know if I could be the leader
   [junit4]   2> 1079131 INFO  (zkCallback-2756-thread-1-processing-n:127.0.0.1:46930_solr) [n:127.0.0.1:46930_solr    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [0])
   [junit4]   2> 1079131 INFO  (zkCallback-2748-thread-1-processing-n:127.0.0.1:54619_solr) [n:127.0.0.1:54619_solr    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [0])
   [junit4]   2> 1079131 INFO  (zkCallback-2757-thread-1-processing-n:127.0.0.1:49958_solr) [n:127.0.0.1:49958_solr    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [0])
   [junit4]   2> 1079131 INFO  (jetty-launcher-2740-thread-3) [n:127.0.0.1:46930_solr    ] o.a.s.c.ZkController Publish node=127.0.0.1:46930_solr as DOWN
   [junit4]   2> 1079132 INFO  (zkCallback-2748-thread-1-processing-n:127.0.0.1:54619_solr) [n:127.0.0.1:54619_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1079132 INFO  (zkCallback-2756-thread-1-processing-n:127.0.0.1:46930_solr) [n:127.0.0.1:46930_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1079134 INFO  (jetty-launcher-2740-thread-3) [n:127.0.0.1:46930_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:46930_solr
   [junit4]   2> 1079134 INFO  (jetty-launcher-2740-thread-3) [n:127.0.0.1:46930_solr    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:46930_solr
   [junit4]   2> 1079135 INFO  (zkCallback-2748-thread-1-processing-n:127.0.0.1:54619_solr) [n:127.0.0.1:54619_solr    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1079135 INFO  (zkCallback-2756-thread-1-processing-n:127.0.0.1:46930_solr) [n:127.0.0.1:46930_solr    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1079135 INFO  (zkCallback-2748-thread-1-processing-n:127.0.0.1:54619_solr) [n:127.0.0.1:54619_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 1079136 INFO  (zkCallback-2756-thread-1-processing-n:127.0.0.1:46930_solr) [n:127.0.0.1:46930_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 1079155 INFO  (zkCallback-2757-thread-1-processing-n:127.0.0.1:49958_solr) [n:127.0.0.1:49958_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1079156 INFO  (jetty-launcher-2740-thread-3) [n:127.0.0.1:46930_solr    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1079156 INFO  (jetty-launcher-2740-thread-3) [n:127.0.0.1:46930_solr    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1079157 INFO  (jetty-launcher-2740-thread-3) [n:127.0.0.1:46930_solr    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.search.stats.TestDistribIDF_1F45A3F87F67AC9C-001/tempDir-001/node3/.
   [junit4]   2> 1079157 INFO  (jetty-launcher-2740-thread-3) [n:127.0.0.1:46930_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions
   [junit4]   2> 1079157 INFO  (zkCallback-2757-thread-2-processing-n:127.0.0.1:49958_solr) [n:127.0.0.1:49958_solr    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1079158 INFO  (zkCallback-2757-thread-2-processing-n:127.0.0.1:49958_solr) [n:127.0.0.1:49958_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 1079158 INFO  (jetty-launcher-2740-thread-3) [n:127.0.0.1:46930_solr    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0
   [junit4]   2> 1079158 INFO  (jetty-launcher-2740-thread-3) [n:127.0.0.1:46930_solr    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1079159 INFO  (jetty-launcher-2740-thread-1) [n:127.0.0.1:54619_solr    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 1079160 INFO  (OverseerStateUpdate-96279825213423621-127.0.0.1:54619_solr-n_0000000000) [n:127.0.0.1:54619_solr    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 1079160 INFO  (OverseerStateUpdate-96279825213423621-127.0.0.1:54619_solr-n_0000000000) [n:127.0.0.1:54619_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (2)
   [junit4]   2> 1079161 INFO  (OverseerStateUpdate-96279825213423621-127.0.0.1:54619_solr-n_0000000000) [n:127.0.0.1:54619_solr    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"downnode",
   [junit4]   2>   "node_name":"127.0.0.1:46930_solr"} current state version: 0
   [junit4]   2> 1079161 INFO  (OverseerStateUpdate-96279825213423621-127.0.0.1:54619_solr-n_0000000000) [n:127.0.0.1:54619_solr    ] o.a.s.c.o.NodeMutator DownNode state invoked for node: 127.0.0.1:46930_solr
   [junit4]   2> 1079163 INFO  (OverseerCollectionConfigSetProcessor-96279825213423621-127.0.0.1:54619_solr-n_0000000000) [n:127.0.0.1:54619_solr    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 1079171 INFO  (jetty-launcher-2740-thread-2) [n:127.0.0.1:49958_solr    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1079171 INFO  (jetty-launcher-2740-thread-2) [n:127.0.0.1:49958_solr    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1079171 INFO  (jetty-launcher-2740-thread-1) [n:127.0.0.1:54619_solr    ] o.a.s.c.ZkController Publish node=127.0.0.1:54619_solr as DOWN
   [junit4]   2> 1079172 INFO  (jetty-launcher-2740-thread-1) [n:127.0.0.1:54619_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:54619_solr
   [junit4]   2> 1079173 INFO  (jetty-launcher-2740-thread-1) [n:127.0.0.1:54619_solr    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:54619_solr
   [junit4]   2> 1079173 INFO  (jetty-launcher-2740-thread-2) [n:127.0.0.1:49958_solr    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.search.stats.TestDistribIDF_1F45A3F87F67AC9C-001/tempDir-001/node2/.
   [junit4]   2> 1079173 INFO  (zkCallback-2756-thread-1-processing-n:127.0.0.1:46930_solr) [n:127.0.0.1:46930_solr    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1079173 INFO  (zkCallback-2757-thread-2-processing-n:127.0.0.1:49958_solr) [n:127.0.0.1:49958_solr    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1079173 INFO  (zkCallback-2748-thread-1-processing-n:127.0.0.1:54619_solr) [n:127.0.0.1:54619_solr    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1079174 INFO  (zkCallback-2757-thread-2-processing-n:127.0.0.1:49958_solr) [n:127.0.0.1:49958_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 1079174 INFO  (zkCallback-2748-thread-1-processing-n:127.0.0.1:54619_solr) [n:127.0.0.1:54619_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 1079173 INFO  (OverseerStateUpdate-96279825213423621-127.0.0.1:54619_solr-n_0000000000) [n:127.0.0.1:54619_solr    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"downnode",
   [junit4]   2>   "node_name":"127.0.0.1:54619_solr"} current state version: 0
   [junit4]   2> 1079174 INFO  (OverseerStateUpdate-96279825213423621-127.0.0.1:54619_solr-n_0000000000) [n:127.0.0.1:54619_solr    ] o.a.s.c.o.NodeMutator DownNode state invoked for node: 127.0.0.1:54619_solr
   [junit4]   2> 1079178 INFO  (zkCallback-2756-thread-1-processing-n:127.0.0.1:46930_solr) [n:127.0.0.1:46930_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 1079180 INFO  (jetty-launcher-2740-thread-2) [n:127.0.0.1:49958_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions
   [junit4]   2> 1079181 INFO  (jetty-launcher-2740-thread-2) [n:127.0.0.1:49958_solr    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0
   [junit4]   2> 1079181 INFO  (jetty-launcher-2740-thread-2) [n:127.0.0.1:49958_solr    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1079208 INFO  (jetty-launcher-2740-thread-1) [n:127.0.0.1:54619_solr    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1079208 INFO  (jetty-launcher-2740-thread-1) [n:127.0.0.1:54619_solr    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1079209 INFO  (jetty-launcher-2740-thread-1) [n:127.0.0.1:54619_solr    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.search.stats.TestDistribIDF_1F45A3F87F67AC9C-001/tempDir-001/node1/.
   [junit4]   2> 1079209 INFO  (jetty-launcher-2740-thread-1) [n:127.0.0.1:54619_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions
   [junit4]   2> 1079210 INFO  (jetty-launcher-2740-thread-1) [n:127.0.0.1:54619_solr    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0
   [junit4]   2> 1079210 INFO  (jetty-launcher-2740-thread-1) [n:127.0.0.1:54619_solr    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1079210 INFO  (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1079214 INFO  (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1079214 INFO  (zkCallback-2760-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@613281a3 name:ZooKeeperConnection Watcher:127.0.0.1:52388 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1079214 INFO  (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1079215 INFO  (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1079216 INFO  (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1079217 INFO  (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1079217 INFO  (zkCallback-2762-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@18785709 name:ZooKeeperConnection Watcher:127.0.0.1:52388/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1079217 INFO  (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1079217 INFO  (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1079218 INFO  (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig-spellcheckcomponent.xml
   [junit4]   2> 1079220 INFO  (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema-null-charfilters-analyzer.xml
   [junit4]   2> 1079221 INFO  (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/regex-boost-processor-test.txt
   [junit4]   2> 1079222 INFO  (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig-spellchecker.xml
   [junit4]   2> 1079223 INFO  (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/bad-schema-multiple-asterisk-copyfield-dest-should-fail-test.xml
   [junit4]   2> 1079225 INFO  (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema-docValuesFaceting.xml
   [junit4]   2> 1079226 INFO  (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig-tlog-with-delayingcomponent.xml
   [junit4]   2> 1079227 INFO  (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/bad-schema-currency-ft-bogus-default-code.xml
   [junit4]   2> 1079228 INFO  (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema-numeric.xml
   [junit4]   2> 1079229 INFO  (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig_codec.xml
   [junit4]   2> 1079230 INFO  (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/bad-schema-multiple-asterisk-copyfield-source-should-fail-test.xml
   [junit4]   2> 1079231 INFO  (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema_latest.xml
   [junit4]   2> 1079233 INFO  (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema-sorts.xml
   [junit4]   2> 1079234 INFO  (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig-headers.xml
   [junit4]   2> 1079235 INFO  (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig-implicitproperties.xml
   [junit4]   2> 1079236 INFO  (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/bad-solrconfig-bogus-scriptengine-name.xml
   [junit4]   2> 1079237 INFO  (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema-snippet-types.incl
   [junit4]   2> 1079238 INFO  (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig-parsing-update-processor-chains.xml
   [junit4]   2> 1079240 INFO  (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema-docValuesMissing.xml
   [junit4]   2> 1079241 INFO  (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig-noopregen.xml
   [junit4]   2> 1079242 INFO  (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema-one-field-no-dynamic-field-unique-key.xml
   [junit4]   2> 1079243 INFO  (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig-script-updateprocessor.xml
   [junit4]   2> 1079244 INFO  (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/bad-error-solrconfig.xml
   [junit4]   2> 1079245 INFO  (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig-suggestercomponent.xml
   [junit4]   2> 1079246 INFO  (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/bad-currency.xml
   [junit4]   2> 1079247 INFO  (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig-query-parser-init.xml
   [junit4]   2> 1079248 INFO  (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig-caching.xml
   [junit4]   2> 1079250 INFO  (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/conditional.updateprocessor.js
   [junit4]   2> 1079251 INFO  (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig-indexconfig-mergepolicyfactory.xml
   [junit4]   2> 1079252 INFO  (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/compoundDictionary.txt
   [junit4]   2> 1079253 INFO  (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig-tieredmergepolicyfactory.xml
   [junit4]   2> 1079254 INFO  (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/phrasesuggest.txt
   [junit4]   2> 1079255 INFO  (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig-transformers.xml
   [junit4]   2> 1079256 INFO  (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/bad-schema-not-indexed-but-norms.xml
   [junit4]   2> 1079257 INFO  (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/blendedInfixSuggest.txt
   [junit4]   2> 1079258 INFO  (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema-phrasesuggest.xml
   [junit4]   2> 1079259 INFO  (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/hunspell-test.dic
   [junit4]   2> 1079260 INFO  (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema-HighlighterMaxOffsetTest.xml
   [junit4]   2> 1079261 INFO  (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema-rest-lucene-match-version.xml
   [junit4]   2> 1079262 INFO  (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig-functionquery.xml
   [junit4]   2> 1079264 INFO  (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig-externalversionconstraint.xml
   [junit4]   2> 1079265 INFO  (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/bad-schema-codec-global-vs-ft-mismatch.xml
   [junit4]   2> 1079266 INFO  (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema-sim.xml
   [junit4]   2> 1079267 INFO  (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema-docValuesMulti.xml
   [junit4]   2> 1079268 INFO  (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig-logmergepolicyfactory.xml
   [junit4]   2> 1079269 INFO  (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig-collapseqparser.xml
   [junit4]   2> 1079270 INFO  (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig-analytics-query.xml
   [junit4]   2> 1079271 INFO  (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema-sweetspot.xml
   [junit4]   2> 1079273 INFO  (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig-searcher-listeners1.xml
   [junit4]   2> 1079274 INFO  (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema-id-and-version-fields-only.xml
   [junit4]   2> 1079275 INFO  (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig-classification.xml
   [junit4]   2> 1079276 INFO  (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/bad-solrconfig-schema-mutable-but-not-managed.xml
   [junit4]   2> 1079277 INFO  (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/hunspell-test.aff
   [junit4]   2> 1079278 INFO  (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig-response-log-component.xml
   [junit4]   2> 1079279 INFO  (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/bad-schema-sim-default-does-not-exist.xml
   [junit4]   2> 1079280 INFO  (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/bad-schema-dynamicfield-required.xml
   [junit4]   2> 1079282 INFO  (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/bad-schema-not-indexed-but-pos.xml
   [junit4]   2> 1079283 INFO  (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig-hash.xml
   [junit4]   2> 1079284 INFO  (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig-components-name.xml
   [junit4]   2> 1079285 INFO  (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/bad-mp-solrconfig.xml
   [junit4]   2> 1079286 INFO  (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1079287 INFO  (TEST-TestDistribIDF.testMultiCollectionQuery-seed#[1F45A3F87F67AC9C]) [

[...truncated too long message...]

0.0-M20.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/bcprov-jdk15-1.45.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/cglib-nodep-2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/commons-collections-3.2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/commons-math3-3.4.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/easymock-3.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/ehcache-core-2.4.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/hadoop-common-2.7.2-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/hadoop-hdfs-2.7.2-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/hadoop-minikdc-2.7.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/htrace-core-3.2.0-incubating.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jackson-annotations-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jackson-core-asl-1.9.13.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jackson-databind-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jackson-mapper-asl-1.9.13.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jersey-core-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jersey-server-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jetty-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jetty-util-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/mina-core-2.0.0-M5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/mockito-core-1.9.5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/netty-all-4.0.36.Final.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/objenesis-1.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/presto-parser-0.122.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/slice-0.10.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/analysis/icu/lucene-analyzers-icu-6.2.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/contrib/solr-analysis-extras/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/queryparser/classes/test:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/contrib/analysis-extras/lib/icu4j-56.1.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-launcher.jar:/x1/jenkins/.ant/lib/ivy-2.3.0.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-antlr.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-regexp.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-resolver.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-net.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jmf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-oro.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-xalan2.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-logging.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jsch.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit4.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-javamail.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jai.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jdepend.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bsf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-swing.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-netrexx.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bcel.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-testutil.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-log4j.jar:/x1/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.8/lib/tools.jar:/x1/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.3.4.jar com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/temp/junit4-J0-20160721_144740_155.events @/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/temp/junit4-J0-20160721_144740_155.suites -stdin
   [junit4] ERROR: JVM J0 ended with an exception: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1501)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:121)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:955)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:952)
   [junit4] 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4] 	at java.lang.Thread.run(Thread.java:745)

BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:771: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:707: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/common-build.xml:536: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/common-build.xml:1443: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/common-build.xml:984: At least one slave process threw an exception, first: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.

Total time: 334 minutes 53 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
ERROR: Failed to archive {README.txt=README.txt, heapdumps/java_pid27116.hprof=heapdumps/java_pid27116.hprof, solr/build/solr-core/test/temp/junit4-J1-20160721_144740_155.events=solr/build/solr-core/test/temp/junit4-J1-20160721_144740_155.events, solr/build/solr-core/test/temp/junit4-J2-20160721_144740_160.events=solr/build/solr-core/test/temp/junit4-J2-20160721_144740_160.events, heapdumps/java_pid3660.hprof=heapdumps/java_pid3660.hprof, solr/build/solr-core/test/temp/junit4-J0-20160721_144740_155.events=solr/build/solr-core/test/temp/junit4-J0-20160721_144740_155.events} due to internal error; falling back to full archiving
java.lang.NullPointerException
	at jsync.protocol.ChangeInputStream.read(ChangeInputStream.java:69)
	at com.cloudbees.jenkins.plugins.jsync.archiver.MD5DigestingInputStream.read(MD5DigestingInputStream.java:39)
	at com.google.common.io.LimitInputStream.read(LimitInputStream.java:79)
	at java.io.FilterInputStream.read(FilterInputStream.java:107)
	at com.google.common.io.ByteStreams.copy(ByteStreams.java:193)
	at jsync.protocol.FileSequenceReader.read(FileSequenceReader.java:35)
	at com.cloudbees.jenkins.plugins.jsync.archiver.JSyncArtifactManager.remoteSync(JSyncArtifactManager.java:148)
	at com.cloudbees.jenkins.plugins.jsync.archiver.JSyncArtifactManager.archive(JSyncArtifactManager.java:72)
	at hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:236)
	at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:78)
	at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
	at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:782)
	at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:723)
	at hudson.model.Build$BuildExecution.post2(Build.java:185)
	at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:668)
	at hudson.model.Run.execute(Run.java:1763)
	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
	at hudson.model.ResourceController.execute(ResourceController.java:98)
	at hudson.model.Executor.run(Executor.java:410)
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any




[JENKINS] Lucene-Solr-NightlyTests-6.x - Build # 124 - Unstable

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-6.x/124/

7 tests failed.
FAILED:  org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.test

Error Message:
Timeout occured while waiting response from server at: http://127.0.0.1:36301/uoh

Stack Trace:
org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:36301/uoh
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:601)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:259)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:248)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.makeRequest(CollectionsAPIDistributedZkTest.java:399)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.deletePartiallyCreatedCollection(CollectionsAPIDistributedZkTest.java:231)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test(CollectionsAPIDistributedZkTest.java:181)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.net.SocketTimeoutException: Read timed out
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
	at java.net.SocketInputStream.read(SocketInputStream.java:170)
	at java.net.SocketInputStream.read(SocketInputStream.java:141)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:160)
	at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:84)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:273)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
	at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
	at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)
	at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)
	at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197)
	at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)
	at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)
	at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685)
	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487)
	at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:495)
	... 47 more


FAILED:  junit.framework.TestSuite.org.apache.solr.handler.TestReplicationHandler

Error Message:
ObjectTracker found 3 object(s) that were not released!!! [NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory]

Stack Trace:
java.lang.AssertionError: ObjectTracker found 3 object(s) that were not released!!! [NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory]
	at __randomizedtesting.SeedInfo.seed([A5BA906E554EE54F]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.junit.Assert.assertNull(Assert.java:551)
	at org.apache.solr.SolrTestCaseJ4.teardownTestCases(SolrTestCaseJ4.java:258)
	at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:834)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  junit.framework.TestSuite.org.apache.solr.hadoop.MorphlineBasicMiniMRTest

Error Message:
java.lang.NoClassDefFoundError: org/jboss/netty/channel/ChannelFactory

Stack Trace:
org.apache.hadoop.yarn.exceptions.YarnRuntimeException: java.lang.NoClassDefFoundError: org/jboss/netty/channel/ChannelFactory
	at __randomizedtesting.SeedInfo.seed([9FDE6A9426A7591B]:0)
	at org.apache.solr.hadoop.hack.MiniYARNCluster$NodeManagerWrapper.serviceStart(MiniYARNCluster.java:334)
	at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
	at org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:120)
	at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
	at org.apache.solr.hadoop.hack.MiniMRClientClusterFactory.create(MiniMRClientClusterFactory.java:83)
	at org.apache.solr.hadoop.hack.MiniMRCluster.<init>(MiniMRCluster.java:181)
	at org.apache.solr.hadoop.hack.MiniMRCluster.<init>(MiniMRCluster.java:169)
	at org.apache.solr.hadoop.MorphlineBasicMiniMRTest.setupClass(MorphlineBasicMiniMRTest.java:180)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:811)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.NoClassDefFoundError: org/jboss/netty/channel/ChannelFactory
	at java.lang.Class.forName0(Native Method)
	at java.lang.Class.forName(Class.java:348)
	at org.apache.hadoop.conf.Configuration.getClassByNameOrNull(Configuration.java:2134)
	at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2099)
	at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2193)
	at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2219)
	at org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices.serviceInit(AuxServices.java:121)
	at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
	at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:107)
	at org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl.serviceInit(ContainerManagerImpl.java:245)
	at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
	at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:107)
	at org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceInit(NodeManager.java:260)
	at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
	at org.apache.solr.hadoop.hack.MiniYARNCluster$NodeManagerWrapper.serviceStart(MiniYARNCluster.java:316)
	... 31 more
Caused by: java.lang.ClassNotFoundException: org.jboss.netty.channel.ChannelFactory
	at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
	... 46 more


FAILED:  junit.framework.TestSuite.org.apache.solr.hadoop.MorphlineBasicMiniMRTest

Error Message:
32 threads leaked from SUITE scope at org.apache.solr.hadoop.MorphlineBasicMiniMRTest:     1) Thread[id=118, name=nioEventLoopGroup-5-3, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)         at java.lang.Thread.run(Thread.java:745)    2) Thread[id=69, name=nioEventLoopGroup-2-7, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)         at java.lang.Thread.run(Thread.java:745)    3) Thread[id=108, name=nioEventLoopGroup-4-1, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)         at java.lang.Thread.run(Thread.java:745)    4) Thread[id=110, name=nioEventLoopGroup-4-3, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)         at java.lang.Thread.run(Thread.java:745)    5) Thread[id=122, name=nioEventLoopGroup-5-7, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)         at java.lang.Thread.run(Thread.java:745)    6) Thread[id=109, name=nioEventLoopGroup-4-2, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)         at java.lang.Thread.run(Thread.java:745)    7) Thread[id=72, name=nioEventLoopGroup-3-2, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)         at java.lang.Thread.run(Thread.java:745)    8) Thread[id=119, name=nioEventLoopGroup-5-4, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)         at java.lang.Thread.run(Thread.java:745)    9) Thread[id=112, name=nioEventLoopGroup-4-5, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)         at java.lang.Thread.run(Thread.java:745)   10) Thread[id=111, name=nioEventLoopGroup-4-4, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)         at java.lang.Thread.run(Thread.java:745)   11) Thread[id=66, name=nioEventLoopGroup-2-4, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)         at java.lang.Thread.run(Thread.java:745)   12) Thread[id=115, name=nioEventLoopGroup-4-8, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)         at java.lang.Thread.run(Thread.java:745)   13) Thread[id=74, name=nioEventLoopGroup-3-4, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)         at java.lang.Thread.run(Thread.java:745)   14) Thread[id=116, name=nioEventLoopGroup-5-1, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)         at java.lang.Thread.run(Thread.java:745)   15) Thread[id=121, name=nioEventLoopGroup-5-6, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)         at java.lang.Thread.run(Thread.java:745)   16) Thread[id=71, name=nioEventLoopGroup-3-1, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)         at java.lang.Thread.run(Thread.java:745)   17) Thread[id=75, name=nioEventLoopGroup-3-5, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)         at java.lang.Thread.run(Thread.java:745)   18) Thread[id=113, name=nioEventLoopGroup-4-6, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)         at java.lang.Thread.run(Thread.java:745)   19) Thread[id=68, name=nioEventLoopGroup-2-6, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)         at java.lang.Thread.run(Thread.java:745)   20) Thread[id=120, name=nioEventLoopGroup-5-5, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)         at java.lang.Thread.run(Thread.java:745)   21) Thread[id=123, name=nioEventLoopGroup-5-8, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)         at java.lang.Thread.run(Thread.java:745)   22) Thread[id=73, name=nioEventLoopGroup-3-3, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)         at java.lang.Thread.run(Thread.java:745)   23) Thread[id=64, name=nioEventLoopGroup-2-2, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)         at java.lang.Thread.run(Thread.java:745)   24) Thread[id=114, name=nioEventLoopGroup-4-7, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)         at java.lang.Thread.run(Thread.java:745)   25) Thread[id=67, name=nioEventLoopGroup-2-5, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)         at java.lang.Thread.run(Thread.java:745)   26) Thread[id=76, name=nioEventLoopGroup-3-6, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)         at java.lang.Thread.run(Thread.java:745)   27) Thread[id=63, name=nioEventLoopGroup-2-1, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)         at java.lang.Thread.run(Thread.java:745)   28) Thread[id=117, name=nioEventLoopGroup-5-2, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)         at java.lang.Thread.run(Thread.java:745)   29) Thread[id=70, name=nioEventLoopGroup-2-8, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)         at java.lang.Thread.run(Thread.java:745)   30) Thread[id=77, name=nioEventLoopGroup-3-7, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)         at java.lang.Thread.run(Thread.java:745)   31) Thread[id=78, name=nioEventLoopGroup-3-8, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)         at java.lang.Thread.run(Thread.java:745)   32) Thread[id=65, name=nioEventLoopGroup-2-3, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)         at java.lang.Thread.run(Thread.java:745)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 32 threads leaked from SUITE scope at org.apache.solr.hadoop.MorphlineBasicMiniMRTest: 
   1) Thread[id=118, name=nioEventLoopGroup-5-3, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]
        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
        at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)
        at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)
        at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
        at java.lang.Thread.run(Thread.java:745)
   2) Thread[id=69, name=nioEventLoopGroup-2-7, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]
        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
        at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)
        at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)
        at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
        at java.lang.Thread.run(Thread.java:745)
   3) Thread[id=108, name=nioEventLoopGroup-4-1, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]
        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
        at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)
        at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)
        at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
        at java.lang.Thread.run(Thread.java:745)
   4) Thread[id=110, name=nioEventLoopGroup-4-3, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]
        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
        at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)
        at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)
        at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
        at java.lang.Thread.run(Thread.java:745)
   5) Thread[id=122, name=nioEventLoopGroup-5-7, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]
        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
        at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)
        at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)
        at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
        at java.lang.Thread.run(Thread.java:745)
   6) Thread[id=109, name=nioEventLoopGroup-4-2, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]
        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
        at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)
        at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)
        at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
        at java.lang.Thread.run(Thread.java:745)
   7) Thread[id=72, name=nioEventLoopGroup-3-2, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]
        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
        at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)
        at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)
        at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
        at java.lang.Thread.run(Thread.java:745)
   8) Thread[id=119, name=nioEventLoopGroup-5-4, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]
        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
        at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)
        at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)
        at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
        at java.lang.Thread.run(Thread.java:745)
   9) Thread[id=112, name=nioEventLoopGroup-4-5, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]
        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
        at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)
        at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)
        at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
        at java.lang.Thread.run(Thread.java:745)
  10) Thread[id=111, name=nioEventLoopGroup-4-4, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]
        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
        at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)
        at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)
        at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
        at java.lang.Thread.run(Thread.java:745)
  11) Thread[id=66, name=nioEventLoopGroup-2-4, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]
        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
        at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)
        at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)
        at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
        at java.lang.Thread.run(Thread.java:745)
  12) Thread[id=115, name=nioEventLoopGroup-4-8, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]
        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
        at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)
        at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)
        at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
        at java.lang.Thread.run(Thread.java:745)
  13) Thread[id=74, name=nioEventLoopGroup-3-4, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]
        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
        at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)
        at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)
        at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
        at java.lang.Thread.run(Thread.java:745)
  14) Thread[id=116, name=nioEventLoopGroup-5-1, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]
        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
        at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)
        at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)
        at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
        at java.lang.Thread.run(Thread.java:745)
  15) Thread[id=121, name=nioEventLoopGroup-5-6, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]
        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
        at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)
        at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)
        at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
        at java.lang.Thread.run(Thread.java:745)
  16) Thread[id=71, name=nioEventLoopGroup-3-1, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]
        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
        at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)
        at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)
        at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
        at java.lang.Thread.run(Thread.java:745)
  17) Thread[id=75, name=nioEventLoopGroup-3-5, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]
        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
        at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)
        at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)
        at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
        at java.lang.Thread.run(Thread.java:745)
  18) Thread[id=113, name=nioEventLoopGroup-4-6, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]
        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
        at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)
        at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)
        at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
        at java.lang.Thread.run(Thread.java:745)
  19) Thread[id=68, name=nioEventLoopGroup-2-6, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]
        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
        at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)
        at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)
        at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
        at java.lang.Thread.run(Thread.java:745)
  20) Thread[id=120, name=nioEventLoopGroup-5-5, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]
        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
        at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)
        at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)
        at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
        at java.lang.Thread.run(Thread.java:745)
  21) Thread[id=123, name=nioEventLoopGroup-5-8, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]
        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
        at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)
        at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)
        at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
        at java.lang.Thread.run(Thread.java:745)
  22) Thread[id=73, name=nioEventLoopGroup-3-3, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]
        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
        at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)
        at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)
        at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
        at java.lang.Thread.run(Thread.java:745)
  23) Thread[id=64, name=nioEventLoopGroup-2-2, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]
        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
        at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)
        at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)
        at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
        at java.lang.Thread.run(Thread.java:745)
  24) Thread[id=114, name=nioEventLoopGroup-4-7, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]
        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
        at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)
        at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)
        at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
        at java.lang.Thread.run(Thread.java:745)
  25) Thread[id=67, name=nioEventLoopGroup-2-5, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]
        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
        at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)
        at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)
        at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
        at java.lang.Thread.run(Thread.java:745)
  26) Thread[id=76, name=nioEventLoopGroup-3-6, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]
        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
        at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)
        at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)
        at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
        at java.lang.Thread.run(Thread.java:745)
  27) Thread[id=63, name=nioEventLoopGroup-2-1, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]
        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
        at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)
        at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)
        at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
        at java.lang.Thread.run(Thread.java:745)
  28) Thread[id=117, name=nioEventLoopGroup-5-2, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]
        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
        at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)
        at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)
        at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
        at java.lang.Thread.run(Thread.java:745)
  29) Thread[id=70, name=nioEventLoopGroup-2-8, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]
        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
        at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)
        at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)
        at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
        at java.lang.Thread.run(Thread.java:745)
  30) Thread[id=77, name=nioEventLoopGroup-3-7, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]
        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
        at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)
        at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)
        at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
        at java.lang.Thread.run(Thread.java:745)
  31) Thread[id=78, name=nioEventLoopGroup-3-8, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]
        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
        at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)
        at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)
        at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
        at java.lang.Thread.run(Thread.java:745)
  32) Thread[id=65, name=nioEventLoopGroup-2-3, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]
        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
        at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)
        at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)
        at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
        at java.lang.Thread.run(Thread.java:745)
	at __randomizedtesting.SeedInfo.seed([9FDE6A9426A7591B]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.hadoop.MorphlineGoLiveMiniMRTest

Error Message:
java.lang.NoClassDefFoundError: org/jboss/netty/channel/ChannelPipelineFactory

Stack Trace:
org.apache.hadoop.yarn.exceptions.YarnRuntimeException: java.lang.NoClassDefFoundError: org/jboss/netty/channel/ChannelPipelineFactory
	at __randomizedtesting.SeedInfo.seed([9FDE6A9426A7591B]:0)
	at org.apache.solr.hadoop.hack.MiniYARNCluster$NodeManagerWrapper.serviceStart(MiniYARNCluster.java:334)
	at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
	at org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:120)
	at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
	at org.apache.solr.hadoop.hack.MiniMRClientClusterFactory.create(MiniMRClientClusterFactory.java:83)
	at org.apache.solr.hadoop.hack.MiniMRClientClusterFactory.create(MiniMRClientClusterFactory.java:39)
	at org.apache.solr.hadoop.MorphlineGoLiveMiniMRTest.setupClass(MorphlineGoLiveMiniMRTest.java:194)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:811)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.NoClassDefFoundError: org/jboss/netty/channel/ChannelPipelineFactory
	at java.lang.Class.forName0(Native Method)
	at java.lang.Class.forName(Class.java:348)
	at org.apache.hadoop.conf.Configuration.getClassByNameOrNull(Configuration.java:2134)
	at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2099)
	at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2193)
	at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2219)
	at org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices.serviceInit(AuxServices.java:121)
	at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
	at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:107)
	at org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl.serviceInit(ContainerManagerImpl.java:245)
	at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
	at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:107)
	at org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceInit(NodeManager.java:260)
	at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
	at org.apache.solr.hadoop.hack.MiniYARNCluster$NodeManagerWrapper.serviceStart(MiniYARNCluster.java:316)
	... 30 more
Caused by: java.lang.ClassNotFoundException: org.jboss.netty.channel.ChannelPipelineFactory
	at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
	... 45 more


FAILED:  junit.framework.TestSuite.org.apache.solr.hadoop.MorphlineGoLiveMiniMRTest

Error Message:
32 threads leaked from SUITE scope at org.apache.solr.hadoop.MorphlineGoLiveMiniMRTest:     1) Thread[id=115, name=nioEventLoopGroup-4-8, state=RUNNABLE, group=TGRP-MorphlineGoLiveMiniMRTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)         at java.lang.Thread.run(Thread.java:745)    2) Thread[id=118, name=nioEventLoopGroup-5-3, state=RUNNABLE, group=TGRP-MorphlineGoLiveMiniMRTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)         at java.lang.Thread.run(Thread.java:745)    3) Thread[id=114, name=nioEventLoopGroup-4-7, state=RUNNABLE, group=TGRP-MorphlineGoLiveMiniMRTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)         at java.lang.Thread.run(Thread.java:745)    4) Thread[id=122, name=nioEventLoopGroup-5-7, state=RUNNABLE, group=TGRP-MorphlineGoLiveMiniMRTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)         at java.lang.Thread.run(Thread.java:745)    5) Thread[id=75, name=nioEventLoopGroup-3-5, state=RUNNABLE, group=TGRP-MorphlineGoLiveMiniMRTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)         at java.lang.Thread.run(Thread.java:745)    6) Thread[id=112, name=nioEventLoopGroup-4-5, state=RUNNABLE, group=TGRP-MorphlineGoLiveMiniMRTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)         at java.lang.Thread.run(Thread.java:745)    7) Thread[id=78, name=nioEventLoopGroup-3-8, state=RUNNABLE, group=TGRP-MorphlineGoLiveMiniMRTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)         at java.lang.Thread.run(Thread.java:745)    8) Thread[id=69, name=nioEventLoopGroup-2-7, state=RUNNABLE, group=TGRP-MorphlineGoLiveMiniMRTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)         at java.lang.Thread.run(Thread.java:745)    9) Thread[id=113, name=nioEventLoopGroup-4-6, state=RUNNABLE, group=TGRP-MorphlineGoLiveMiniMRTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)         at java.lang.Thread.run(Thread.java:745)   10) Thread[id=110, name=nioEventLoopGroup-4-3, state=RUNNABLE, group=TGRP-MorphlineGoLiveMiniMRTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)         at java.lang.Thread.run(Thread.java:745)   11) Thread[id=111, name=nioEventLoopGroup-4-4, state=RUNNABLE, group=TGRP-MorphlineGoLiveMiniMRTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)         at java.lang.Thread.run(Thread.java:745)   12) Thread[id=74, name=nioEventLoopGroup-3-4, state=RUNNABLE, group=TGRP-MorphlineGoLiveMiniMRTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)         at java.lang.Thread.run(Thread.java:745)   13) Thread[id=116, name=nioEventLoopGroup-5-1, state=RUNNABLE, group=TGRP-MorphlineGoLiveMiniMRTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)         at java.lang.Thread.run(Thread.java:745)   14) Thread[id=65, name=nioEventLoopGroup-2-3, state=RUNNABLE, group=TGRP-MorphlineGoLiveMiniMRTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)         at java.lang.Thread.run(Thread.java:745)   15) Thread[id=72, name=nioEventLoopGroup-3-2, state=RUNNABLE, group=TGRP-MorphlineGoLiveMiniMRTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)         at java.lang.Thread.run(Thread.java:745)   16) Thread[id=108, name=nioEventLoopGroup-4-1, state=RUNNABLE, group=TGRP-MorphlineGoLiveMiniMRTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThrea

[...truncated too long message...]

io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)
   [junit4]    >         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >   24) Thread[id=73, name=nioEventLoopGroup-3-3, state=RUNNABLE, group=TGRP-MorphlineGoLiveMiniMRTest]
   [junit4]    >         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
   [junit4]    >         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
   [junit4]    >         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
   [junit4]    >         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
   [junit4]    >         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
   [junit4]    >         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)
   [junit4]    >         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)
   [junit4]    >         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)
   [junit4]    >         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >   25) Thread[id=63, name=nioEventLoopGroup-2-1, state=RUNNABLE, group=TGRP-MorphlineGoLiveMiniMRTest]
   [junit4]    >         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
   [junit4]    >         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
   [junit4]    >         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
   [junit4]    >         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
   [junit4]    >         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
   [junit4]    >         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)
   [junit4]    >         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)
   [junit4]    >         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)
   [junit4]    >         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >   26) Thread[id=109, name=nioEventLoopGroup-4-2, state=RUNNABLE, group=TGRP-MorphlineGoLiveMiniMRTest]
   [junit4]    >         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
   [junit4]    >         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
   [junit4]    >         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
   [junit4]    >         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
   [junit4]    >         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
   [junit4]    >         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)
   [junit4]    >         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)
   [junit4]    >         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)
   [junit4]    >         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >   27) Thread[id=71, name=nioEventLoopGroup-3-1, state=RUNNABLE, group=TGRP-MorphlineGoLiveMiniMRTest]
   [junit4]    >         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
   [junit4]    >         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
   [junit4]    >         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
   [junit4]    >         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
   [junit4]    >         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
   [junit4]    >         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)
   [junit4]    >         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)
   [junit4]    >         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)
   [junit4]    >         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >   28) Thread[id=117, name=nioEventLoopGroup-5-2, state=RUNNABLE, group=TGRP-MorphlineGoLiveMiniMRTest]
   [junit4]    >         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
   [junit4]    >         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
   [junit4]    >         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
   [junit4]    >         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
   [junit4]    >         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
   [junit4]    >         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)
   [junit4]    >         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)
   [junit4]    >         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)
   [junit4]    >         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >   29) Thread[id=76, name=nioEventLoopGroup-3-6, state=RUNNABLE, group=TGRP-MorphlineGoLiveMiniMRTest]
   [junit4]    >         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
   [junit4]    >         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
   [junit4]    >         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
   [junit4]    >         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
   [junit4]    >         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
   [junit4]    >         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)
   [junit4]    >         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)
   [junit4]    >         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)
   [junit4]    >         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >   30) Thread[id=67, name=nioEventLoopGroup-2-5, state=RUNNABLE, group=TGRP-MorphlineGoLiveMiniMRTest]
   [junit4]    >         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
   [junit4]    >         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
   [junit4]    >         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
   [junit4]    >         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
   [junit4]    >         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
   [junit4]    >         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)
   [junit4]    >         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)
   [junit4]    >         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)
   [junit4]    >         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >   31) Thread[id=120, name=nioEventLoopGroup-5-5, state=RUNNABLE, group=TGRP-MorphlineGoLiveMiniMRTest]
   [junit4]    >         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
   [junit4]    >         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
   [junit4]    >         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
   [junit4]    >         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
   [junit4]    >         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
   [junit4]    >         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)
   [junit4]    >         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)
   [junit4]    >         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)
   [junit4]    >         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >   32) Thread[id=64, name=nioEventLoopGroup-2-2, state=RUNNABLE, group=TGRP-MorphlineGoLiveMiniMRTest]
   [junit4]    >         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
   [junit4]    >         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
   [junit4]    >         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
   [junit4]    >         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
   [junit4]    >         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
   [junit4]    >         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)
   [junit4]    >         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)
   [junit4]    >         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)
   [junit4]    >         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([9FDE6A9426A7591B]:0)
   [junit4] Completed [7/7 (2!)] on J1 in 26.62s, 0 tests, 2 errors <<< FAILURES!

[...truncated 52204 lines...]



[JENKINS] Lucene-Solr-NightlyTests-6.x - Build # 123 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-6.x/123/

6 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.hadoop.MorphlineBasicMiniMRTest

Error Message:
java.lang.NoClassDefFoundError: org/jboss/netty/channel/ChannelFactory

Stack Trace:
org.apache.hadoop.yarn.exceptions.YarnRuntimeException: java.lang.NoClassDefFoundError: org/jboss/netty/channel/ChannelFactory
	at __randomizedtesting.SeedInfo.seed([A8C544380102C1BF]:0)
	at org.apache.solr.hadoop.hack.MiniYARNCluster$NodeManagerWrapper.serviceStart(MiniYARNCluster.java:334)
	at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
	at org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:120)
	at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
	at org.apache.solr.hadoop.hack.MiniMRClientClusterFactory.create(MiniMRClientClusterFactory.java:83)
	at org.apache.solr.hadoop.hack.MiniMRCluster.<init>(MiniMRCluster.java:181)
	at org.apache.solr.hadoop.hack.MiniMRCluster.<init>(MiniMRCluster.java:169)
	at org.apache.solr.hadoop.MorphlineBasicMiniMRTest.setupClass(MorphlineBasicMiniMRTest.java:180)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:811)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.NoClassDefFoundError: org/jboss/netty/channel/ChannelFactory
	at java.lang.Class.forName0(Native Method)
	at java.lang.Class.forName(Class.java:348)
	at org.apache.hadoop.conf.Configuration.getClassByNameOrNull(Configuration.java:2134)
	at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2099)
	at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2193)
	at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2219)
	at org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices.serviceInit(AuxServices.java:121)
	at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
	at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:107)
	at org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl.serviceInit(ContainerManagerImpl.java:245)
	at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
	at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:107)
	at org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceInit(NodeManager.java:260)
	at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
	at org.apache.solr.hadoop.hack.MiniYARNCluster$NodeManagerWrapper.serviceStart(MiniYARNCluster.java:316)
	... 31 more
Caused by: java.lang.ClassNotFoundException: org.jboss.netty.channel.ChannelFactory
	at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
	... 46 more


FAILED:  junit.framework.TestSuite.org.apache.solr.hadoop.MorphlineBasicMiniMRTest

Error Message:
32 threads leaked from SUITE scope at org.apache.solr.hadoop.MorphlineBasicMiniMRTest:     1) Thread[id=111, name=nioEventLoopGroup-4-2, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)         at java.lang.Thread.run(Thread.java:745)    2) Thread[id=75, name=nioEventLoopGroup-3-5, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)         at java.lang.Thread.run(Thread.java:745)    3) Thread[id=122, name=nioEventLoopGroup-5-5, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)         at java.lang.Thread.run(Thread.java:745)    4) Thread[id=63, name=nioEventLoopGroup-2-1, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)         at java.lang.Thread.run(Thread.java:745)    5) Thread[id=117, name=nioEventLoopGroup-4-8, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)         at java.lang.Thread.run(Thread.java:745)    6) Thread[id=68, name=nioEventLoopGroup-2-6, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)         at java.lang.Thread.run(Thread.java:745)    7) Thread[id=65, name=nioEventLoopGroup-2-3, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)         at java.lang.Thread.run(Thread.java:745)    8) Thread[id=66, name=nioEventLoopGroup-2-4, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)         at java.lang.Thread.run(Thread.java:745)    9) Thread[id=125, name=nioEventLoopGroup-5-8, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)         at java.lang.Thread.run(Thread.java:745)   10) Thread[id=118, name=nioEventLoopGroup-5-1, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)         at java.lang.Thread.run(Thread.java:745)   11) Thread[id=120, name=nioEventLoopGroup-5-3, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)         at java.lang.Thread.run(Thread.java:745)   12) Thread[id=116, name=nioEventLoopGroup-4-7, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)         at java.lang.Thread.run(Thread.java:745)   13) Thread[id=72, name=nioEventLoopGroup-3-2, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)         at java.lang.Thread.run(Thread.java:745)   14) Thread[id=115, name=nioEventLoopGroup-4-6, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)         at java.lang.Thread.run(Thread.java:745)   15) Thread[id=124, name=nioEventLoopGroup-5-7, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)         at java.lang.Thread.run(Thread.java:745)   16) Thread[id=110, name=nioEventLoopGroup-4-1, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)         at java.lang.Thread.run(Thread.java:745)   17) Thread[id=70, name=nioEventLoopGroup-2-8, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)         at java.lang.Thread.run(Thread.java:745)   18) Thread[id=77, name=nioEventLoopGroup-3-7, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)         at java.lang.Thread.run(Thread.java:745)   19) Thread[id=64, name=nioEventLoopGroup-2-2, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)         at java.lang.Thread.run(Thread.java:745)   20) Thread[id=71, name=nioEventLoopGroup-3-1, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)         at java.lang.Thread.run(Thread.java:745)   21) Thread[id=113, name=nioEventLoopGroup-4-4, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)         at java.lang.Thread.run(Thread.java:745)   22) Thread[id=74, name=nioEventLoopGroup-3-4, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)         at java.lang.Thread.run(Thread.java:745)   23) Thread[id=67, name=nioEventLoopGroup-2-5, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)         at java.lang.Thread.run(Thread.java:745)   24) Thread[id=69, name=nioEventLoopGroup-2-7, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)         at java.lang.Thread.run(Thread.java:745)   25) Thread[id=112, name=nioEventLoopGroup-4-3, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)         at java.lang.Thread.run(Thread.java:745)   26) Thread[id=114, name=nioEventLoopGroup-4-5, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)         at java.lang.Thread.run(Thread.java:745)   27) Thread[id=121, name=nioEventLoopGroup-5-4, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)         at java.lang.Thread.run(Thread.java:745)   28) Thread[id=76, name=nioEventLoopGroup-3-6, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)         at java.lang.Thread.run(Thread.java:745)   29) Thread[id=78, name=nioEventLoopGroup-3-8, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)         at java.lang.Thread.run(Thread.java:745)   30) Thread[id=119, name=nioEventLoopGroup-5-2, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)         at java.lang.Thread.run(Thread.java:745)   31) Thread[id=123, name=nioEventLoopGroup-5-6, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)         at java.lang.Thread.run(Thread.java:745)   32) Thread[id=73, name=nioEventLoopGroup-3-3, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)         at java.lang.Thread.run(Thread.java:745)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 32 threads leaked from SUITE scope at org.apache.solr.hadoop.MorphlineBasicMiniMRTest: 
   1) Thread[id=111, name=nioEventLoopGroup-4-2, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]
        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
        at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)
        at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)
        at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
        at java.lang.Thread.run(Thread.java:745)
   2) Thread[id=75, name=nioEventLoopGroup-3-5, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]
        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
        at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)
        at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)
        at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
        at java.lang.Thread.run(Thread.java:745)
   3) Thread[id=122, name=nioEventLoopGroup-5-5, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]
        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
        at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)
        at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)
        at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
        at java.lang.Thread.run(Thread.java:745)
   4) Thread[id=63, name=nioEventLoopGroup-2-1, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]
        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
        at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)
        at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)
        at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
        at java.lang.Thread.run(Thread.java:745)
   5) Thread[id=117, name=nioEventLoopGroup-4-8, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]
        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
        at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)
        at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)
        at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
        at java.lang.Thread.run(Thread.java:745)
   6) Thread[id=68, name=nioEventLoopGroup-2-6, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]
        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
        at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)
        at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)
        at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
        at java.lang.Thread.run(Thread.java:745)
   7) Thread[id=65, name=nioEventLoopGroup-2-3, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]
        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
        at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)
        at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)
        at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
        at java.lang.Thread.run(Thread.java:745)
   8) Thread[id=66, name=nioEventLoopGroup-2-4, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]
        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
        at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)
        at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)
        at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
        at java.lang.Thread.run(Thread.java:745)
   9) Thread[id=125, name=nioEventLoopGroup-5-8, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]
        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
        at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)
        at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)
        at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
        at java.lang.Thread.run(Thread.java:745)
  10) Thread[id=118, name=nioEventLoopGroup-5-1, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]
        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
        at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)
        at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)
        at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
        at java.lang.Thread.run(Thread.java:745)
  11) Thread[id=120, name=nioEventLoopGroup-5-3, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]
        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
        at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)
        at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)
        at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
        at java.lang.Thread.run(Thread.java:745)
  12) Thread[id=116, name=nioEventLoopGroup-4-7, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]
        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
        at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)
        at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)
        at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
        at java.lang.Thread.run(Thread.java:745)
  13) Thread[id=72, name=nioEventLoopGroup-3-2, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]
        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
        at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)
        at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)
        at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
        at java.lang.Thread.run(Thread.java:745)
  14) Thread[id=115, name=nioEventLoopGroup-4-6, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]
        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
        at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)
        at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)
        at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
        at java.lang.Thread.run(Thread.java:745)
  15) Thread[id=124, name=nioEventLoopGroup-5-7, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]
        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
        at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)
        at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)
        at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
        at java.lang.Thread.run(Thread.java:745)
  16) Thread[id=110, name=nioEventLoopGroup-4-1, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]
        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
        at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)
        at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)
        at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
        at java.lang.Thread.run(Thread.java:745)
  17) Thread[id=70, name=nioEventLoopGroup-2-8, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]
        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
        at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)
        at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)
        at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
        at java.lang.Thread.run(Thread.java:745)
  18) Thread[id=77, name=nioEventLoopGroup-3-7, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]
        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
        at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)
        at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)
        at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
        at java.lang.Thread.run(Thread.java:745)
  19) Thread[id=64, name=nioEventLoopGroup-2-2, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]
        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
        at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)
        at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)
        at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
        at java.lang.Thread.run(Thread.java:745)
  20) Thread[id=71, name=nioEventLoopGroup-3-1, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]
        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
        at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)
        at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)
        at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
        at java.lang.Thread.run(Thread.java:745)
  21) Thread[id=113, name=nioEventLoopGroup-4-4, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]
        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
        at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)
        at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)
        at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
        at java.lang.Thread.run(Thread.java:745)
  22) Thread[id=74, name=nioEventLoopGroup-3-4, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]
        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
        at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)
        at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)
        at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
        at java.lang.Thread.run(Thread.java:745)
  23) Thread[id=67, name=nioEventLoopGroup-2-5, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]
        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
        at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)
        at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)
        at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
        at java.lang.Thread.run(Thread.java:745)
  24) Thread[id=69, name=nioEventLoopGroup-2-7, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]
        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
        at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)
        at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)
        at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
        at java.lang.Thread.run(Thread.java:745)
  25) Thread[id=112, name=nioEventLoopGroup-4-3, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]
        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
        at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)
        at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)
        at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
        at java.lang.Thread.run(Thread.java:745)
  26) Thread[id=114, name=nioEventLoopGroup-4-5, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]
        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
        at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)
        at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)
        at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
        at java.lang.Thread.run(Thread.java:745)
  27) Thread[id=121, name=nioEventLoopGroup-5-4, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]
        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
        at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)
        at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)
        at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
        at java.lang.Thread.run(Thread.java:745)
  28) Thread[id=76, name=nioEventLoopGroup-3-6, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]
        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
        at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)
        at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)
        at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
        at java.lang.Thread.run(Thread.java:745)
  29) Thread[id=78, name=nioEventLoopGroup-3-8, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]
        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
        at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)
        at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)
        at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
        at java.lang.Thread.run(Thread.java:745)
  30) Thread[id=119, name=nioEventLoopGroup-5-2, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]
        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
        at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)
        at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)
        at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
        at java.lang.Thread.run(Thread.java:745)
  31) Thread[id=123, name=nioEventLoopGroup-5-6, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]
        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
        at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)
        at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)
        at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
        at java.lang.Thread.run(Thread.java:745)
  32) Thread[id=73, name=nioEventLoopGroup-3-3, state=RUNNABLE, group=TGRP-MorphlineBasicMiniMRTest]
        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
        at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)
        at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)
        at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
        at java.lang.Thread.run(Thread.java:745)
	at __randomizedtesting.SeedInfo.seed([A8C544380102C1BF]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.hadoop.MorphlineGoLiveMiniMRTest

Error Message:
java.lang.NoClassDefFoundError: org/jboss/netty/channel/ChannelFactory

Stack Trace:
org.apache.hadoop.yarn.exceptions.YarnRuntimeException: java.lang.NoClassDefFoundError: org/jboss/netty/channel/ChannelFactory
	at __randomizedtesting.SeedInfo.seed([A8C544380102C1BF]:0)
	at org.apache.solr.hadoop.hack.MiniYARNCluster$NodeManagerWrapper.serviceStart(MiniYARNCluster.java:334)
	at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
	at org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:120)
	at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
	at org.apache.solr.hadoop.hack.MiniMRClientClusterFactory.create(MiniMRClientClusterFactory.java:83)
	at org.apache.solr.hadoop.hack.MiniMRClientClusterFactory.create(MiniMRClientClusterFactory.java:39)
	at org.apache.solr.hadoop.MorphlineGoLiveMiniMRTest.setupClass(MorphlineGoLiveMiniMRTest.java:194)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:811)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.NoClassDefFoundError: org/jboss/netty/channel/ChannelFactory
	at java.lang.Class.forName0(Native Method)
	at java.lang.Class.forName(Class.java:348)
	at org.apache.hadoop.conf.Configuration.getClassByNameOrNull(Configuration.java:2134)
	at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2099)
	at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2193)
	at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2219)
	at org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices.serviceInit(AuxServices.java:121)
	at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
	at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:107)
	at org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl.serviceInit(ContainerManagerImpl.java:245)
	at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
	at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:107)
	at org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceInit(NodeManager.java:260)
	at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
	at org.apache.solr.hadoop.hack.MiniYARNCluster$NodeManagerWrapper.serviceStart(MiniYARNCluster.java:316)
	... 30 more
Caused by: java.lang.ClassNotFoundException: org.jboss.netty.channel.ChannelFactory
	at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
	... 45 more


FAILED:  junit.framework.TestSuite.org.apache.solr.hadoop.MorphlineGoLiveMiniMRTest

Error Message:
32 threads leaked from SUITE scope at org.apache.solr.hadoop.MorphlineGoLiveMiniMRTest:     1) Thread[id=113, name=nioEventLoopGroup-4-5, state=RUNNABLE, group=TGRP-MorphlineGoLiveMiniMRTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)         at java.lang.Thread.run(Thread.java:745)    2) Thread[id=63, name=nioEventLoopGroup-2-1, state=RUNNABLE, group=TGRP-MorphlineGoLiveMiniMRTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)         at java.lang.Thread.run(Thread.java:745)    3) Thread[id=124, name=nioEventLoopGroup-5-8, state=RUNNABLE, group=TGRP-MorphlineGoLiveMiniMRTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)         at java.lang.Thread.run(Thread.java:745)    4) Thread[id=117, name=nioEventLoopGroup-5-1, state=RUNNABLE, group=TGRP-MorphlineGoLiveMiniMRTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)         at java.lang.Thread.run(Thread.java:745)    5) Thread[id=122, name=nioEventLoopGroup-5-6, state=RUNNABLE, group=TGRP-MorphlineGoLiveMiniMRTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)         at java.lang.Thread.run(Thread.java:745)    6) Thread[id=70, name=nioEventLoopGroup-2-8, state=RUNNABLE, group=TGRP-MorphlineGoLiveMiniMRTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)         at java.lang.Thread.run(Thread.java:745)    7) Thread[id=71, name=nioEventLoopGroup-3-1, state=RUNNABLE, group=TGRP-MorphlineGoLiveMiniMRTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)         at java.lang.Thread.run(Thread.java:745)    8) Thread[id=78, name=nioEventLoopGroup-3-8, state=RUNNABLE, group=TGRP-MorphlineGoLiveMiniMRTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)         at java.lang.Thread.run(Thread.java:745)    9) Thread[id=69, name=nioEventLoopGroup-2-7, state=RUNNABLE, group=TGRP-MorphlineGoLiveMiniMRTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)         at java.lang.Thread.run(Thread.java:745)   10) Thread[id=72, name=nioEventLoopGroup-3-2, state=RUNNABLE, group=TGRP-MorphlineGoLiveMiniMRTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)         at java.lang.Thread.run(Thread.java:745)   11) Thread[id=116, name=nioEventLoopGroup-4-8, state=RUNNABLE, group=TGRP-MorphlineGoLiveMiniMRTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)         at java.lang.Thread.run(Thread.java:745)   12) Thread[id=118, name=nioEventLoopGroup-5-2, state=RUNNABLE, group=TGRP-MorphlineGoLiveMiniMRTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)         at java.lang.Thread.run(Thread.java:745)   13) Thread[id=109, name=nioEventLoopGroup-4-1, state=RUNNABLE, group=TGRP-MorphlineGoLiveMiniMRTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)         at java.lang.Thread.run(Thread.java:745)   14) Thread[id=77, name=nioEventLoopGroup-3-7, state=RUNNABLE, group=TGRP-MorphlineGoLiveMiniMRTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)         at java.lang.Thread.run(Thread.java:745)   15) Thread[id=65, name=nioEventLoopGroup-2-3, state=RUNNABLE, group=TGRP-MorphlineGoLiveMiniMRTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)         at java.lang.Thread.run(Thread.java:745)   16) Thread[id=67, name=nioEventLoopGroup-2-5, state=RUNNABLE, group=TGRP-MorphlineGoLiveMiniMRTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)         at java.lang.Thread.run(Thread.java:745)   17) Thread[id=74, name=nioEventLoopGroup-3-4, state=RUNNABLE, group=TGRP-MorphlineGoLiveMiniMRTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)         at java.lang.Thread.run(Thread.java:745)   18) Thread[id=75, name=nioEventLoopGroup-3-5, state=RUNNABLE, group=TGRP-MorphlineGoLiveMiniMRTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)         at java.lang.Thread.run(Thread.java:745)   19) Thread[id=115, name=nioEventLoopGroup-4-7, state=RUNNABLE, group=TGRP-MorphlineGoLiveMiniMRTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)         at java.lang.Thread.run(Thread.java:745)   20) Thread[id=121, name=nioEventLoopGroup-5-5, state=RUNNABLE, group=TGRP-MorphlineGoLiveMiniMRTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)         at java.lang.Thread.run(Thread.java:745)   21) Thread[id=112, name=nioEventLoopGroup-4-4, state=RUNNABLE, group=TGRP-MorphlineGoLiveMiniMRTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)         at java.lang.Thread.run(Thread.java:745)   22) Thread[id=64, name=nioEventLoopGroup-2-2, state=RUNNABLE, group=TGRP-MorphlineGoLiveMiniMRTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)         at java.lang.Thread.run(Thread.java:745)   23) Thread[id=111, name=nioEventLoopGroup-4-3, state=RUNNABLE, group=TGRP-MorphlineGoLiveMiniMRTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)         at java.lang.Thread.run(Thread.java:745)   24) Thread[id=114, name=nioEventLoopGroup-4-6, state=RUNNABLE, group=TGRP-MorphlineGoLiveMiniMRTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)         at java.lang.Thread.run(Thread.java:745)   25) Thread[id=76, name=nioEventLoopGroup-3-6, state=RUNNABLE, group=TGRP-MorphlineGoLiveMiniMRTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)         at java.lang.Thread.run(Thread.java:745)   26) Thread[id=110, name=nioEventLoopGroup-4-2, state=RUNNABLE, group=TGRP-MorphlineGoLiveMiniMRTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)         at java.lang.Thread.run(Thread.java:745)   27) Thread[id=120, name=nioEventLoopGroup-5-4, state=RUNNABLE, group=TGRP-MorphlineGoLiveMiniMRTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)         at java.lang.Thread.run(Thread.java:745)   28) Thread[id=73, name=nioEventLoopGroup-3-3, state=RUNNABLE, group=TGRP-MorphlineGoLiveMiniMRTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Nativ

[...truncated too long message...]

Select(EPollSelectorImpl.java:93)
   [junit4]    >         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
   [junit4]    >         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
   [junit4]    >         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)
   [junit4]    >         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)
   [junit4]    >         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)
   [junit4]    >         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >   25) Thread[id=76, name=nioEventLoopGroup-3-6, state=RUNNABLE, group=TGRP-MorphlineGoLiveMiniMRTest]
   [junit4]    >         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
   [junit4]    >         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
   [junit4]    >         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
   [junit4]    >         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
   [junit4]    >         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
   [junit4]    >         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)
   [junit4]    >         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)
   [junit4]    >         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)
   [junit4]    >         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >   26) Thread[id=110, name=nioEventLoopGroup-4-2, state=RUNNABLE, group=TGRP-MorphlineGoLiveMiniMRTest]
   [junit4]    >         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
   [junit4]    >         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
   [junit4]    >         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
   [junit4]    >         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
   [junit4]    >         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
   [junit4]    >         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)
   [junit4]    >         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)
   [junit4]    >         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)
   [junit4]    >         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >   27) Thread[id=120, name=nioEventLoopGroup-5-4, state=RUNNABLE, group=TGRP-MorphlineGoLiveMiniMRTest]
   [junit4]    >         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
   [junit4]    >         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
   [junit4]    >         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
   [junit4]    >         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
   [junit4]    >         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
   [junit4]    >         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)
   [junit4]    >         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)
   [junit4]    >         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)
   [junit4]    >         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >   28) Thread[id=73, name=nioEventLoopGroup-3-3, state=RUNNABLE, group=TGRP-MorphlineGoLiveMiniMRTest]
   [junit4]    >         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
   [junit4]    >         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
   [junit4]    >         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
   [junit4]    >         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
   [junit4]    >         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
   [junit4]    >         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)
   [junit4]    >         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)
   [junit4]    >         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)
   [junit4]    >         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >   29) Thread[id=119, name=nioEventLoopGroup-5-3, state=RUNNABLE, group=TGRP-MorphlineGoLiveMiniMRTest]
   [junit4]    >         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
   [junit4]    >         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
   [junit4]    >         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
   [junit4]    >         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
   [junit4]    >         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
   [junit4]    >         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)
   [junit4]    >         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)
   [junit4]    >         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)
   [junit4]    >         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >   30) Thread[id=123, name=nioEventLoopGroup-5-7, state=RUNNABLE, group=TGRP-MorphlineGoLiveMiniMRTest]
   [junit4]    >         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
   [junit4]    >         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
   [junit4]    >         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
   [junit4]    >         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
   [junit4]    >         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
   [junit4]    >         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)
   [junit4]    >         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)
   [junit4]    >         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)
   [junit4]    >         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >   31) Thread[id=68, name=nioEventLoopGroup-2-6, state=RUNNABLE, group=TGRP-MorphlineGoLiveMiniMRTest]
   [junit4]    >         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
   [junit4]    >         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
   [junit4]    >         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
   [junit4]    >         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
   [junit4]    >         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
   [junit4]    >         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)
   [junit4]    >         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)
   [junit4]    >         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)
   [junit4]    >         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >   32) Thread[id=66, name=nioEventLoopGroup-2-4, state=RUNNABLE, group=TGRP-MorphlineGoLiveMiniMRTest]
   [junit4]    >         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
   [junit4]    >         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
   [junit4]    >         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
   [junit4]    >         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
   [junit4]    >         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
   [junit4]    >         at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:639)
   [junit4]    >         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:325)
   [junit4]    >         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)
   [junit4]    >         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([A8C544380102C1BF]:0)
   [junit4] Completed [7/7 (2!)] on J1 in 29.01s, 0 tests, 2 errors <<< FAILURES!

[...truncated 3165 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:771: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:715: Some of the tests produced a heap dump, but did not fail. Maybe a suppressed OutOfMemoryError? Dumps created:
* java_pid17874.hprof

Total time: 714 minutes 4 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
No prior successful build to compare, so performing full copy of artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any




[JENKINS] Lucene-Solr-NightlyTests-6.x - Build # 122 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-6.x/122/

All tests passed

Build Log:
[...truncated 10504 lines...]
    [javac] Compiling 939 source files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/classes/java
    [javac] /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/java/org/apache/solr/core/CoreSorter.java:162: error: cannot find symbol
    [javac]     return cc.getNodeConfig().getNodeName();
    [javac]              ^
    [javac]   symbol:   method getNodeConfig()
    [javac]   location: variable cc of type CoreContainer
    [javac] /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/java/org/apache/solr/core/CoreContainer.java:466: error: method getCoreLoadThreadCount in class NodeConfig cannot be applied to given types;
    [javac]         ( zkSys.getZkController() == null ? cfg.getCoreLoadThreadCount() : Integer.MAX_VALUE ),
    [javac]                                                ^
    [javac]   required: int
    [javac]   found: no arguments
    [javac]   reason: actual and formal argument lists differ in length
    [javac] Note: Some input files use or override a deprecated API.
    [javac] Note: Recompile with -Xlint:deprecation for details.
    [javac] Note: Some input files use unchecked or unsafe operations.
    [javac] Note: Recompile with -Xlint:unchecked for details.
    [javac] 2 errors

BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:770: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:707: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/common-build.xml:536: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/common-build.xml:484: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/common-build.xml:385: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/common-build.xml:501: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/common-build.xml:1955: Compile failed; see the compiler error output for details.

Total time: 240 minutes 25 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
No prior successful build to compare, so performing full copy of artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any