You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2016/04/30 03:49:10 UTC

[JENKINS] Lucene-Solr-NightlyTests-6.x - Build # 52 - Still Failing

Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-6.x/52/

3 tests failed.
FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=1094, name=collection4, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=1094, name=collection4, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
	at __randomizedtesting.SeedInfo.seed([D87C2A1E874D0DFE:502815C429B16006]:0)
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:55263: collection already exists: awholynewstresscollection_collection4_5
	at __randomizedtesting.SeedInfo.seed([D87C2A1E874D0DFE]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:590)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:259)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:248)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:404)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:357)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1192)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:962)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:898)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1595)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1616)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:987)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.TestTolerantUpdateProcessorRandomCloud

Error Message:
Could not find collection:test_col

Stack Trace:
java.lang.AssertionError: Could not find collection:test_col
	at __randomizedtesting.SeedInfo.seed([D87C2A1E874D0DFE]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.junit.Assert.assertNotNull(Assert.java:526)
	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:150)
	at org.apache.solr.cloud.TestTolerantUpdateProcessorRandomCloud.createMiniSolrCloudCluster(TestTolerantUpdateProcessorRandomCloud.java:135)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:811)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.handler.TestReplicationHandler.doTestStressReplication

Error Message:
timed out waiting for collection1 startAt time to exceed: Sat Apr 30 07:51:23 ICT 2016

Stack Trace:
java.lang.AssertionError: timed out waiting for collection1 startAt time to exceed: Sat Apr 30 07:51:23 ICT 2016
	at __randomizedtesting.SeedInfo.seed([D87C2A1E874D0DFE:3D72AD88265644D]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.solr.handler.TestReplicationHandler.watchCoreStartAt(TestReplicationHandler.java:1428)
	at org.apache.solr.handler.TestReplicationHandler.doTestStressReplication(TestReplicationHandler.java:780)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 11310 lines...]
   [junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D87C2A1E874D0DFE-001/init-core-data-001
   [junit4]   2> 65590 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[D87C2A1E874D0DFE]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false)
   [junit4]   2> 65590 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[D87C2A1E874D0DFE]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   2> 65594 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 65595 INFO  (Thread-344) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 65595 INFO  (Thread-344) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 65695 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [    ] o.a.s.c.ZkTestServer start zk server on port:39223
   [junit4]   2> 65695 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 65696 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 65714 INFO  (zkCallback-60-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1c45fb97 name:ZooKeeperConnection Watcher:127.0.0.1:39223 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 65714 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 65714 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 65715 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 65718 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 65730 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 65735 INFO  (zkCallback-61-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@327c3eb4 name:ZooKeeperConnection Watcher:127.0.0.1:39223/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 65735 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 65735 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 65735 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 65737 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 65739 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 65740 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 65741 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 65741 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 65745 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 65745 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 65750 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 65751 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 65753 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 65753 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 65755 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 65755 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 65756 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 65756 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 65757 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 65758 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 65759 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 65759 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 65761 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 65761 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 65765 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 65765 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 65767 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 65767 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 65878 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D87C2A1E874D0DFE-001/control-001/cores/collection1
   [junit4]   2> 65880 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [    ] o.e.j.s.Server jetty-9.3.8.v20160314
   [junit4]   2> 65882 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@23bfc24b{/,null,AVAILABLE}
   [junit4]   2> 65885 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [    ] o.e.j.s.ServerConnector Started ServerConnector@7ec9621e{HTTP/1.1,[http/1.1]}{127.0.0.1:53731}
   [junit4]   2> 65885 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [    ] o.e.j.s.Server Started @68824ms
   [junit4]   2> 65885 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D87C2A1E874D0DFE-001/tempDir-001/control/data, hostContext=/, hostPort=53731, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D87C2A1E874D0DFE-001/control-001/cores}
   [junit4]   2> 65885 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@73d16e93
   [junit4]   2> 65885 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D87C2A1E874D0DFE-001/control-001'
   [junit4]   2> 65885 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 65885 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 65886 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 65886 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 65889 INFO  (zkCallback-62-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@222ef9fb name:ZooKeeperConnection Watcher:127.0.0.1:39223/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 65889 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 65890 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 65892 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 65892 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D87C2A1E874D0DFE-001/control-001/solr.xml
   [junit4]   2> 65899 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D87C2A1E874D0DFE-001/control-001/cores
   [junit4]   2> 65899 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [    ] o.a.s.c.CoreContainer New CoreContainer 3858867
   [junit4]   2> 65899 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D87C2A1E874D0DFE-001/control-001]
   [junit4]   2> 65900 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D87C2A1E874D0DFE-001/control-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D87C2A1E874D0DFE-001/control-001/lib
   [junit4]   2> 65900 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 65901 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 65902 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 65902 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 65902 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:39223/solr
   [junit4]   2> 65902 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 65902 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 65903 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 65906 INFO  (zkCallback-65-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3ddf3a9b name:ZooKeeperConnection Watcher:127.0.0.1:39223 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 65906 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 65907 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 65909 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [n:127.0.0.1:53731_    ] o.a.s.c.ZkController Added new OnReconnect listener org.apache.solr.cloud.ZkController$$Lambda$38/887802213@2304a10c
   [junit4]   2> 65910 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [n:127.0.0.1:53731_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 65912 INFO  (zkCallback-66-thread-1-processing-n:127.0.0.1:53731_) [n:127.0.0.1:53731_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4c9e88c8 name:ZooKeeperConnection Watcher:127.0.0.1:39223/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 65912 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [n:127.0.0.1:53731_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 65915 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [n:127.0.0.1:53731_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 65917 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [n:127.0.0.1:53731_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 65919 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [n:127.0.0.1:53731_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 65921 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [n:127.0.0.1:53731_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 65922 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [n:127.0.0.1:53731_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 65924 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [n:127.0.0.1:53731_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 65927 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [n:127.0.0.1:53731_    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 65928 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [n:127.0.0.1:53731_    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 65929 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [n:127.0.0.1:53731_    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 65930 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [n:127.0.0.1:53731_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 65931 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [n:127.0.0.1:53731_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (0)
   [junit4]   2> 65932 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [n:127.0.0.1:53731_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 65933 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [n:127.0.0.1:53731_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 65934 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [n:127.0.0.1:53731_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 65935 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [n:127.0.0.1:53731_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/95812075095588868-127.0.0.1:53731_-n_0000000000
   [junit4]   2> 65935 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [n:127.0.0.1:53731_    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:53731_
   [junit4]   2> 65935 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [n:127.0.0.1:53731_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 65936 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [n:127.0.0.1:53731_    ] o.a.s.c.Overseer Overseer (id=95812075095588868-127.0.0.1:53731_-n_0000000000) starting
   [junit4]   2> 65938 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [n:127.0.0.1:53731_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 65943 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [n:127.0.0.1:53731_    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 65944 INFO  (OverseerCollectionConfigSetProcessor-95812075095588868-127.0.0.1:53731_-n_0000000000) [n:127.0.0.1:53731_    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 65944 INFO  (OverseerStateUpdate-95812075095588868-127.0.0.1:53731_-n_0000000000) [n:127.0.0.1:53731_    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 65944 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [n:127.0.0.1:53731_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:53731_
   [junit4]   2> 65945 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [n:127.0.0.1:53731_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:53731_
   [junit4]   2> 65946 INFO  (zkCallback-66-thread-1-processing-n:127.0.0.1:53731_) [n:127.0.0.1:53731_    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [0])
   [junit4]   2> 65946 INFO  (OverseerStateUpdate-95812075095588868-127.0.0.1:53731_-n_0000000000) [n:127.0.0.1:53731_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 65949 INFO  (zkCallback-66-thread-1-processing-n:127.0.0.1:53731_) [n:127.0.0.1:53731_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (1)
   [junit4]   2> 65971 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [n:127.0.0.1:53731_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 65971 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [n:127.0.0.1:53731_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 65972 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [n:127.0.0.1:53731_    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D87C2A1E874D0DFE-001/control-001/cores
   [junit4]   2> 65973 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [n:127.0.0.1:53731_    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, loadOnStartup=true, configSetProperties=configsetprops.json, collection=control_collection, coreNodeName=, dataDir=data/, shard=}
   [junit4]   2> 65973 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [n:127.0.0.1:53731_    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D87C2A1E874D0DFE-001/control-001/cores/collection1
   [junit4]   2> 65973 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [n:127.0.0.1:53731_    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 65974 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [n:127.0.0.1:53731_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1
   [junit4]   2> 65974 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [n:127.0.0.1:53731_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 65974 INFO  (coreLoadExecutor-292-thread-1-processing-n:127.0.0.1:53731_) [n:127.0.0.1:53731_ c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 65974 INFO  (coreLoadExecutor-292-thread-1-processing-n:127.0.0.1:53731_) [n:127.0.0.1:53731_ c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 65976 INFO  (coreLoadExecutor-292-thread-1-processing-n:127.0.0.1:53731_) [n:127.0.0.1:53731_ c:control_collection   x:collection1] o.a.s.c.ZkController Collection control_collection not visible yet, but flagging it so a watch is registered when it becomes visible
   [junit4]   2> 65976 INFO  (coreLoadExecutor-292-thread-1-processing-n:127.0.0.1:53731_) [n:127.0.0.1:53731_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader addZkWatch [control_collection]
   [junit4]   2> 65976 INFO  (OverseerStateUpdate-95812075095588868-127.0.0.1:53731_-n_0000000000) [n:127.0.0.1:53731_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:53731",
   [junit4]   2>   "node_name":"127.0.0.1:53731_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 65977 INFO  (OverseerStateUpdate-95812075095588868-127.0.0.1:53731_-n_0000000000) [n:127.0.0.1:53731_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:53731",
   [junit4]   2>   "node_name":"127.0.0.1:53731_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 65977 INFO  (OverseerStateUpdate-95812075095588868-127.0.0.1:53731_-n_0000000000) [n:127.0.0.1:53731_    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 65977 INFO  (coreLoadExecutor-292-thread-1-processing-n:127.0.0.1:53731_) [n:127.0.0.1:53731_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Deleting data for [control_collection]
   [junit4]   2> 65978 INFO  (OverseerStateUpdate-95812075095588868-127.0.0.1:53731_-n_0000000000) [n:127.0.0.1:53731_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 65979 INFO  (coreLoadExecutor-292-thread-1-processing-n:127.0.0.1:53731_) [n:127.0.0.1:53731_ c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 65980 INFO  (zkCallback-66-thread-1-processing-n:127.0.0.1:53731_) [n:127.0.0.1:53731_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 66979 INFO  (coreLoadExecutor-292-thread-1-processing-n:127.0.0.1:53731_) [n:127.0.0.1:53731_ c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 66980 INFO  (coreLoadExecutor-292-thread-1-processing-n:127.0.0.1:53731_) [n:127.0.0.1:53731_ c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 66980 INFO  (coreLoadExecutor-292-thread-1-processing-n:127.0.0.1:53731_) [n:127.0.0.1:53731_ c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 66980 INFO  (coreLoadExecutor-292-thread-1-processing-n:127.0.0.1:53731_) [n:127.0.0.1:53731_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/control_collection]
   [junit4]   2> 66981 INFO  (coreLoadExecutor-292-thread-1-processing-n:127.0.0.1:53731_) [n:127.0.0.1:53731_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/control_collection] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 66981 INFO  (coreLoadExecutor-292-thread-1-processing-n:127.0.0.1:53731_) [n:127.0.0.1:53731_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D87C2A1E874D0DFE-001/control-001/cores/collection1'
   [junit4]   2> 66981 INFO  (coreLoadExecutor-292-thread-1-processing-n:127.0.0.1:53731_) [n:127.0.0.1:53731_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 66981 INFO  (coreLoadExecutor-292-thread-1-processing-n:127.0.0.1:53731_) [n:127.0.0.1:53731_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 66983 INFO  (coreLoadExecutor-292-thread-1-processing-n:127.0.0.1:53731_) [n:127.0.0.1:53731_ c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 66996 INFO  (coreLoadExecutor-292-thread-1-processing-n:127.0.0.1:53731_) [n:127.0.0.1:53731_ c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 67002 WARN  (coreLoadExecutor-292-thread-1-processing-n:127.0.0.1:53731_) [n:127.0.0.1:53731_ c:control_collection   x:collection1] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
   [junit4]   2> 67003 INFO  (coreLoadExecutor-292-thread-1-processing-n:127.0.0.1:53731_) [n:127.0.0.1:53731_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.1.0
   [junit4]   2> 67012 INFO  (coreLoadExecutor-292-thread-1-processing-n:127.0.0.1:53731_) [n:127.0.0.1:53731_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 67018 INFO  (coreLoadExecutor-292-thread-1-processing-n:127.0.0.1:53731_) [n:127.0.0.1:53731_ c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 67111 INFO  (coreLoadExecutor-292-thread-1-processing-n:127.0.0.1:53731_) [n:127.0.0.1:53731_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 67117 INFO  (coreLoadExecutor-292-thread-1-processing-n:127.0.0.1:53731_) [n:127.0.0.1:53731_ c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 67118 INFO  (coreLoadExecutor-292-thread-1-processing-n:127.0.0.1:53731_) [n:127.0.0.1:53731_ c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 67132 INFO  (coreLoadExecutor-292-thread-1-processing-n:127.0.0.1:53731_) [n:127.0.0.1:53731_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 67134 INFO  (coreLoadExecutor-292-thread-1-processing-n:127.0.0.1:53731_) [n:127.0.0.1:53731_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 67136 INFO  (coreLoadExecutor-292-thread-1-processing-n:127.0.0.1:53731_) [n:127.0.0.1:53731_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 67137 INFO  (coreLoadExecutor-292-thread-1-processing-n:127.0.0.1:53731_) [n:127.0.0.1:53731_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 67142 INFO  (coreLoadExecutor-292-thread-1-processing-n:127.0.0.1:53731_) [n:127.0.0.1:53731_ c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1
   [junit4]   2> 67142 INFO  (coreLoadExecutor-292-thread-1-processing-n:127.0.0.1:53731_) [n:127.0.0.1:53731_ c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 67143 INFO  (coreLoadExecutor-292-thread-1-processing-n:127.0.0.1:53731_) [n:127.0.0.1:53731_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 67143 INFO  (coreLoadExecutor-292-thread-1-processing-n:127.0.0.1:53731_) [n:127.0.0.1:53731_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D87C2A1E874D0DFE-001/control-001/cores/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D87C2A1E874D0DFE-001/control-001/cores/collection1/data/]
   [junit4]   2> 67143 INFO  (coreLoadExecutor-292-thread-1-processing-n:127.0.0.1:53731_) [n:127.0.0.1:53731_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@3a7ad154
   [junit4]   2> 67144 INFO  (coreLoadExecutor-292-thread-1-processing-n:127.0.0.1:53731_) [n:127.0.0.1:53731_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D87C2A1E874D0DFE-001/control-001/cores/collection1/data
   [junit4]   2> 67145 INFO  (coreLoadExecutor-292-thread-1-processing-n:127.0.0.1:53731_) [n:127.0.0.1:53731_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D87C2A1E874D0DFE-001/control-001/cores/collection1/data/index/
   [junit4]   2> 67145 WARN  (coreLoadExecutor-292-thread-1-processing-n:127.0.0.1:53731_) [n:127.0.0.1:53731_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D87C2A1E874D0DFE-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 67145 INFO  (coreLoadExecutor-292-thread-1-processing-n:127.0.0.1:53731_) [n:127.0.0.1:53731_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D87C2A1E874D0DFE-001/control-001/cores/collection1/data/index
   [junit4]   2> 67145 INFO  (coreLoadExecutor-292-thread-1-processing-n:127.0.0.1:53731_) [n:127.0.0.1:53731_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=9, maxMergeAtOnceExplicit=6, maxMergedSegmentMB=77.5712890625, floorSegmentMB=1.3564453125, forceMergeDeletesPctAllowed=8.889765143520206, segmentsPerTier=16.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.7228485563731526
   [junit4]   2> 67146 INFO  (coreLoadExecutor-292-thread-1-processing-n:127.0.0.1:53731_) [n:127.0.0.1:53731_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@7ecab59b lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@4c7ce6fe),segFN=segments_1,generation=1}
   [junit4]   2> 67146 INFO  (coreLoadExecutor-292-thread-1-processing-n:127.0.0.1:53731_) [n:127.0.0.1:53731_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 67148 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:53731_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D87C2A1E874D0DFE-001/control-001/cores/collection1/data/
   [junit4]   2> 67149 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:53731_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D87C2A1E874D0DFE-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 67152 INFO  (coreLoadExecutor-292-thread-1-processing-n:127.0.0.1:53731_) [n:127.0.0.1:53731_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 67152 INFO  (coreLoadExecutor-292-thread-1-processing-n:127.0.0.1:53731_) [n:127.0.0.1:53731_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 67152 INFO  (coreLoadExecutor-292-thread-1-processing-n:127.0.0.1:53731_) [n:127.0.0.1:53731_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 67152 INFO  (coreLoadExecutor-292-thread-1-processing-n:127.0.0.1:53731_) [n:127.0.0.1:53731_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 67153 INFO  (coreLoadExecutor-292-thread-1-processing-n:127.0.0.1:53731_) [n:127.0.0.1:53731_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 67153 INFO  (coreLoadExecutor-292-thread-1-processing-n:127.0.0.1:53731_) [n:127.0.0.1:53731_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 67153 INFO  (coreLoadExecutor-292-thread-1-processing-n:127.0.0.1:53731_) [n:127.0.0.1:53731_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 67153 INFO  (coreLoadExecutor-292-thread-1-processing-n:127.0.0.1:53731_) [n:127.0.0.1:53731_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 67153 INFO  (coreLoadExecutor-292-thread-1-processing-n:127.0.0.1:53731_) [n:127.0.0.1:53731_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 67153 WARN  (coreLoadExecutor-292-thread-1-processing-n:127.0.0.1:53731_) [n:127.0.0.1:53731_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
   [junit4]   2> 67155 INFO  (coreLoadExecutor-292-thread-1-processing-n:127.0.0.1:53731_) [n:127.0.0.1:53731_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 67156 INFO  (coreLoadExecutor-292-thread-1-processing-n:127.0.0.1:53731_) [n:127.0.0.1:53731_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 67157 INFO  (coreLoadExecutor-292-thread-1-processing-n:127.0.0.1:53731_) [n:127.0.0.1:53731_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 67158 INFO  (coreLoadExecutor-292-thread-1-processing-n:127.0.0.1:53731_) [n:127.0.0.1:53731_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 67162 INFO  (coreLoadExecutor-292-thread-1-processing-n:127.0.0.1:53731_) [n:127.0.0.1:53731_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/mbeans,standard,/dump,/update/csv,/sql,/update/json/docs,/admin/luke,/export,/admin/segments,/get,/admin/system,/replication,/admin/properties,/config,/stream,/schema,/admin/plugins,/admin/logging,/update/json,/admin/ping,/admin/threads,/update,/admin/file
   [junit4]   2> 67162 INFO  (coreLoadExecutor-292-thread-1-processing-n:127.0.0.1:53731_) [n:127.0.0.1:53731_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 67162 INFO  (coreLoadExecutor-292-thread-1-processing-n:127.0.0.1:53731_) [n:127.0.0.1:53731_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 67162 INFO  (coreLoadExecutor-292-thread-1-processing-n:127.0.0.1:53731_) [n:127.0.0.1:53731_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 67163 INFO  (coreLoadExecutor-292-thread-1-processing-n:127.0.0.1:53731_) [n:127.0.0.1:53731_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 67163 INFO  (coreLoadExecutor-292-thread-1-processing-n:127.0.0.1:53731_) [n:127.0.0.1:53731_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 67165 INFO  (coreLoadExecutor-292-thread-1-processing-n:127.0.0.1:53731_) [n:127.0.0.1:53731_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=45, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.8426334006169524]
   [junit4]   2> 67165 INFO  (coreLoadExecutor-292-thread-1-processing-n:127.0.0.1:53731_) [n:127.0.0.1:53731_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@7ecab59b lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@4c7ce6fe),segFN=segments_1,generation=1}
   [junit4]   2> 67165 INFO  (coreLoadExecutor-292-thread-1-processing-n:127.0.0.1:53731_) [n:127.0.0.1:53731_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 67165 INFO  (coreLoadExecutor-292-thread-1-processing-n:127.0.0.1:53731_) [n:127.0.0.1:53731_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@30712c90[collection1] main]
   [junit4]   2> 67165 INFO  (coreLoadExecutor-292-thread-1-processing-n:127.0.0.1:53731_) [n:127.0.0.1:53731_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/control_collection]
   [junit4]   2> 67168 INFO  (coreLoadExecutor-292-thread-1-processing-n:127.0.0.1:53731_) [n:127.0.0.1:53731_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/control_collection] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 67168 INFO  (coreLoadExecutor-292-thread-1-processing-n:127.0.0.1:53731_) [n:127.0.0.1:53731_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 67168 INFO  (coreLoadExecutor-292-thread-1-processing-n:127.0.0.1:53731_) [n:127.0.0.1:53731_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 67169 INFO  (coreLoadExecutor-292-thread-1-processing-n:127.0.0.1:53731_) [n:127.0.0.1:53731_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 67169 INFO  (coreLoadExecutor-292-thread-1-processing-n:127.0.0.1:53731_) [n:127.0.0.1:53731_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 67169 INFO  (coreLoadExecutor-292-thread-1-processing-n:127.0.0.1:53731_) [n:127.0.0.1:53731_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 67169 INFO  (coreLoadExecutor-292-thread-1-processing-n:127.0.0.1:53731_) [n:127.0.0.1:53731_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 67169 INFO  (coreLoadExecutor-292-thread-1-processing-n:127.0.0.1:53731_) [n:127.0.0.1:53731_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 67169 INFO  (coreLoadExecutor-292-thread-1-processing-n:127.0.0.1:53731_) [n:127.0.0.1:53731_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 67170 INFO  (searcherExecutor-293-thread-1-processing-n:127.0.0.1:53731_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:53731_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@30712c90[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 67171 INFO  (coreLoadExecutor-292-thread-1-processing-n:127.0.0.1:53731_) [n:127.0.0.1:53731_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 67171 INFO  (coreLoadExecutor-292-thread-1-processing-n:127.0.0.1:53731_) [n:127.0.0.1:53731_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 67171 INFO  (coreLoadExecutor-292-thread-1-processing-n:127.0.0.1:53731_) [n:127.0.0.1:53731_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 67171 INFO  (coreLoadExecutor-292-thread-1-processing-n:127.0.0.1:53731_) [n:127.0.0.1:53731_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1532993203175686144
   [junit4]   2> 67172 INFO  (coreLoadExecutor-292-thread-1-processing-n:127.0.0.1:53731_) [n:127.0.0.1:53731_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 1.0ms to seed version buckets with highest version 1532993203175686144
   [junit4]   2> 67172 INFO  (coreLoadExecutor-292-thread-1-processing-n:127.0.0.1:53731_) [n:127.0.0.1:53731_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 67173 INFO  (coreLoadExecutor-292-thread-1-processing-n:127.0.0.1:53731_) [n:127.0.0.1:53731_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 67173 INFO  (coreZkRegister-286-thread-1-processing-n:127.0.0.1:53731_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:53731_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:53731 collection:control_collection shard:shard1
   [junit4]   2> 67174 INFO  (coreZkRegister-286-thread-1-processing-n:127.0.0.1:53731_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:53731_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 67178 INFO  (coreZkRegister-286-thread-1-processing-n:127.0.0.1:53731_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:53731_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/95812075095588868-core_node1-n_0000000000
   [junit4]   2> 67179 INFO  (coreZkRegister-286-thread-1-processing-n:127.0.0.1:53731_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:53731_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard=shard1 and weAreReplacement=false and leaderVoteWait=180000
   [junit4]   2> 67181 INFO  (coreZkRegister-286-thread-1-processing-n:127.0.0.1:53731_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:53731_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 67181 INFO  (coreZkRegister-286-thread-1-processing-n:127.0.0.1:53731_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:53731_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 67181 INFO  (coreZkRegister-286-thread-1-processing-n:127.0.0.1:53731_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:53731_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:53731/collection1/
   [junit4]   2> 67181 INFO  (coreZkRegister-286-thread-1-processing-n:127.0.0.1:53731_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:53731_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 67181 INFO  (coreZkRegister-286-thread-1-processing-n:127.0.0.1:53731_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:53731_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:53731/collection1/ has no replicas
   [junit4]   2> 67182 INFO  (coreZkRegister-286-thread-1-processing-n:127.0.0.1:53731_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:53731_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 67184 INFO  (coreZkRegister-286-thread-1-processing-n:127.0.0.1:53731_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:53731_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/control_collection/leaders/shard1/leader after winning as /collections/control_collection/leader_elect/shard1/election/95812075095588868-core_node1-n_0000000000
   [junit4]   2> 67184 INFO  (OverseerStateUpdate-95812075095588868-127.0.0.1:53731_-n_0000000000) [n:127.0.0.1:53731_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 67188 INFO  (coreZkRegister-286-thread-1-processing-n:127.0.0.1:53731_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:53731_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:53731/collection1/ shard1
   [junit4]   2> 67188 INFO  (OverseerStateUpdate-95812075095588868-127.0.0.1:53731_-n_0000000000) [n:127.0.0.1:53731_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:53731",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 1
   [junit4]   2> 67291 INFO  (zkCallback-66-thread-1-processing-n:127.0.0.1:53731_) [n:127.0.0.1:53731_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 67339 INFO  (coreZkRegister-286-thread-1-processing-n:127.0.0.1:53731_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:53731_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:53731/collection1/ and leader is http://127.0.0.1:53731/collection1/
   [junit4]   2> 67339 INFO  (coreZkRegister-286-thread-1-processing-n:127.0.0.1:53731_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:53731_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:53731
   [junit4]   2> 67339 INFO  (coreZkRegister-286-thread-1-processing-n:127.0.0.1:53731_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:53731_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 67340 INFO  (coreZkRegister-286-thread-1-processing-n:127.0.0.1:53731_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:53731_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 67340 INFO  (coreZkRegister-286-thread-1-processing-n:127.0.0.1:53731_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:53731_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 67342 INFO  (OverseerStateUpdate-95812075095588868-127.0.0.1:53731_-n_0000000000) [n:127.0.0.1:53731_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:53731",
   [junit4]   2>   "node_name":"127.0.0.1:53731_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"} current state version: 2
   [junit4]   2> 67343 INFO  (OverseerStateUpdate-95812075095588868-127.0.0.1:53731_-n_0000000000) [n:127.0.0.1:53731_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:53731",
   [junit4]   2>   "node_name":"127.0.0.1:53731_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 67446 INFO  (zkCallback-66-thread-1-processing-n:127.0.0.1:53731_) [n:127.0.0.1:53731_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 67476 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 67477 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 67479 INFO  (zkCallback-68-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7c3d14d8 name:ZooKeeperConnection Watcher:127.0.0.1:39223/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 67479 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 67479 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 67479 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 67485 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 67486 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 67595 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D87C2A1E874D0DFE-001/shard-1-001/cores/collection1
   [junit4]   2> 67595 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D87C2A1E874D0DFE-001/shard-1-001
   [junit4]   2> 67596 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [    ] o.e.j.s.Server jetty-9.3.8.v20160314
   [junit4]   2> 67598 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@2f928fd0{/,null,AVAILABLE}
   [junit4]   2> 67598 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [    ] o.e.j.s.ServerConnector Started ServerConnector@48131c0d{HTTP/1.1,[http/1.1]}{127.0.0.1:40003}
   [junit4]   2> 67599 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [    ] o.e.j.s.Server Started @70538ms
   [junit4]   2> 67599 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D87C2A1E874D0DFE-001/tempDir-001/jetty1, solrconfig=solrconfig.xml, hostContext=/, hostPort=40003, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D87C2A1E874D0DFE-001/shard-1-001/cores}
   [junit4]   2> 67599 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@73d16e93
   [junit4]   2> 67599 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D87C2A1E874D0DFE-001/shard-1-001'
   [junit4]   2> 67599 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 67599 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 67600 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 67600 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 67606 INFO  (zkCallback-69-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2930bb33 name:ZooKeeperConnection Watcher:127.0.0.1:39223/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 67606 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 67606 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 67610 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 67610 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D87C2A1E874D0DFE-001/shard-1-001/solr.xml
   [junit4]   2> 67615 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D87C2A1E874D0DFE-001/shard-1-001/cores
   [junit4]   2> 67615 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [    ] o.a.s.c.CoreContainer New CoreContainer 1650000496
   [junit4]   2> 67615 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D87C2A1E874D0DFE-001/shard-1-001]
   [junit4]   2> 67616 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D87C2A1E874D0DFE-001/shard-1-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D87C2A1E874D0DFE-001/shard-1-001/lib
   [junit4]   2> 67616 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 67617 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 67618 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 67618 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 67618 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:39223/solr
   [junit4]   2> 67618 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 67618 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 67623 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 67628 INFO  (zkCallback-72-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@42eff37a name:ZooKeeperConnection Watcher:127.0.0.1:39223 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 67629 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 67630 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 67631 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [n:127.0.0.1:40003_    ] o.a.s.c.ZkController Added new OnReconnect listener org.apache.solr.cloud.ZkController$$Lambda$38/887802213@6c184750
   [junit4]   2> 67636 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [n:127.0.0.1:40003_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 67643 INFO  (zkCallback-73-thread-1-processing-n:127.0.0.1:40003_) [n:127.0.0.1:40003_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6852de2 name:ZooKeeperConnection Watcher:127.0.0.1:39223/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 67643 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [n:127.0.0.1:40003_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 67649 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [n:127.0.0.1:40003_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 67650 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [n:127.0.0.1:40003_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 67652 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [n:127.0.0.1:40003_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 67652 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [n:127.0.0.1:40003_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/95812075095588872-127.0.0.1:40003_-n_0000000001
   [junit4]   2> 67653 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [n:127.0.0.1:40003_    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/95812075095588868-127.0.0.1:53731_-n_0000000000 to know if I could be the leader
   [junit4]   2> 67653 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [n:127.0.0.1:40003_    ] o.a.s.c.ZkController Publish node=127.0.0.1:40003_ as DOWN
   [junit4]   2> 67655 INFO  (OverseerStateUpdate-95812075095588868-127.0.0.1:53731_-n_0000000000) [n:127.0.0.1:53731_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"downnode",
   [junit4]   2>   "node_name":"127.0.0.1:40003_"} current state version: 3
   [junit4]   2> 67655 INFO  (OverseerStateUpdate-95812075095588868-127.0.0.1:53731_-n_0000000000) [n:127.0.0.1:53731_    ] o.a.s.c.o.NodeMutator DownNode state invoked for node: 127.0.0.1:40003_
   [junit4]   2> 67758 INFO  (zkCallback-68-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 67758 INFO  (zkCallback-66-thread-1-processing-n:127.0.0.1:53731_) [n:127.0.0.1:53731_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 67758 INFO  (zkCallback-73-thread-1-processing-n:127.0.0.1:40003_) [n:127.0.0.1:40003_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 68655 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [n:127.0.0.1:40003_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:40003_
   [junit4]   2> 68656 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [n:127.0.0.1:40003_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:40003_
   [junit4]   2> 68658 INFO  (zkCallback-73-thread-1-processing-n:127.0.0.1:40003_) [n:127.0.0.1:40003_    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 68660 INFO  (zkCallback-66-thread-1-processing-n:127.0.0.1:53731_) [n:127.0.0.1:53731_    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 68660 INFO  (zkCallback-68-thread-1) [    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 68660 INFO  (zkCallback-68-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 68660 INFO  (zkCallback-73-thread-1-processing-n:127.0.0.1:40003_) [n:127.0.0.1:40003_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 68668 INFO  (zkCallback-66-thread-1-processing-n:127.0.0.1:53731_) [n:127.0.0.1:53731_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 68669 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [n:127.0.0.1:40003_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 68670 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [n:127.0.0.1:40003_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 68670 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [n:127.0.0.1:40003_    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D87C2A1E874D0DFE-001/shard-1-001/cores
   [junit4]   2> 68671 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [n:127.0.0.1:40003_    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, loadOnStartup=true, configSetProperties=configsetprops.json, collection=collection1, coreNodeName=, dataDir=data/, shard=}
   [junit4]   2> 68671 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [n:127.0.0.1:40003_    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D87C2A1E874D0DFE-001/shard-1-001/cores/collection1
   [junit4]   2> 68671 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [n:127.0.0.1:40003_    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 68673 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [n:127.0.0.1:40003_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1
   [junit4]   2> 68673 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D87C2A1E874D0DFE]) [n:127.0.0.1:40003_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 68674 INFO  (coreLoadExecutor-303-thread-1-processing-n:127.0.0.1:40003_) [n:127.0.0.1:40003_ c:collection1   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 68674 INFO  (coreLoadExecutor-303-thread-1-processing-n:127.0.0.1:40003_) [n:127.0.0.1:40003_ c:collection1   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 68675 INFO  (coreLoadExecutor-303-thread-1-processing-n:127.0.0.1:40003_) [n:127.0.0.1:40003_ c:collection1   x:collection1] o.a.s.c.ZkController Collection collection1 not visible yet, but flagging it so a watch is registered when it becomes visible
   [junit4]   2> 68675 INFO  (coreLoadExecutor-303-thread-1-processing-n:127.0.0.1:40003_) [n:127.0.0.1:40003_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader addZkWatch [collection1]
   [junit4]   2> 68675 INFO  (OverseerStateUpdate-95812075095588868-127.0.0.1:53731_-n_0000000000) [n:127.0.0.1:53731_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:40003",
   [junit4]   2>   "node_name":"127.0.0.1:40003_",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "operation":"state"} current state version: 4
   [junit4]   2> 68675 INFO  (coreLoadExecutor-303-thread-1-processing-n:127.0.0.1:40003_) [n:127.0.0.1:40003_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Deleting data for [collection1]
   [junit4]   2> 68675 INFO  (OverseerStateUpdate-95812075095588868-127.0.0.1:53731_-n_0000000000) [n:127.0.0.1:53731_    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:40003",
   [junit4]   2>   "node_name":"127.0.0.1:40003_",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 68675 INFO  (coreLoadExecutor-303-thread-1-processing-n:127.0.0.1:40003_) [n:127.0.0.1:40003_ c:collection1   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 68676 INFO  (OverseerStateUpdate-95812075095588868-127.0.0.1:53731_-n_0000000000) [n:127.0.0.1:53731_    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 68676 INFO  (OverseerStateUpdate-95812075095588868-127.0.0.1:53731_-n_0000000000) [n:127.0.0.1:53731_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2
   [junit4]   2> 68778 INFO  (zkCallback-73-thread-1-processing-n:127.0.0.1:40003_) [n:127.0.0.1:40003_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [2])
   [junit4]   2> 68778 INFO  (zkCallback-66-thread-1-processing-n:127.0.0.1:53731_) [n:127.0.0.1:53731_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [2])
   [junit4]   2> 68778 INFO  (zkCallback-68-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [2])
   [junit4]   2> 69676 INFO  (coreLoadExecutor-303-thread-1-processing-n:127.0.0.1:40003_) [n:127.0.0.1:40003_ c:collection1   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 69676 INFO  (coreLoadExecutor-303-thread-1-processing-n:127.0.0.1:40003_) [n:127.0.0.1:40003_ c:collection1   x:colle

[...truncated too long message...]

ess$200(LeaderElector.java:56)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector$ElectionWatcher.process(LeaderElector.java:348)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$3.lambda$process$0(SolrZkClient.java:266)
   [junit4]   2> 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
   [junit4]   2> 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
   [junit4]   2> 	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 3930538 WARN  (zkCallback-3794-thread-2-processing-n:127.0.0.1:39387_solr) [n:127.0.0.1:39387_solr    ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
   [junit4]   2> 3930536 WARN  (zkCallback-3817-thread-2-processing-n:127.0.0.1:42884_solr) [n:127.0.0.1:42884_solr    ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
   [junit4]   2> 3930538 WARN  (zkCallback-3815-thread-2-processing-n:127.0.0.1:46941_solr) [n:127.0.0.1:46941_solr    ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
   [junit4]   2> 3930537 INFO  (jetty-launcher-3775-thread-13) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@3359f584{/solr,null,UNAVAILABLE}
   [junit4]   2> 3930539 INFO  (jetty-launcher-3775-thread-7) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@2883f82c{/solr,null,UNAVAILABLE}
   [junit4]   2> 3930539 INFO  (jetty-launcher-3775-thread-1) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@654d6eda{/solr,null,UNAVAILABLE}
   [junit4]   2> 3930539 INFO  (jetty-launcher-3775-thread-8) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@78bf24a3{/solr,null,UNAVAILABLE}
   [junit4]   2> 3930540 INFO  (jetty-launcher-3775-thread-6) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@3fb36723{/solr,null,UNAVAILABLE}
   [junit4]   2> 3930543 INFO  (SUITE-TestTolerantUpdateProcessorRandomCloud-seed#[D87C2A1E874D0DFE]-worker) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:34152 34152
   [junit4]   2> 3930721 INFO  (Thread-100393) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:34152 34152
   [junit4]   2> 3930722 WARN  (Thread-100393) [    ] o.a.s.c.ZkTestServer Watch limit violations: 
   [junit4]   2> Maximum concurrent create/delete watches above limit:
   [junit4]   2> 
   [junit4]   2> 	14	/solr/aliases.json
   [junit4]   2> 	13	/solr/security.json
   [junit4]   2> 	11	/solr/configs/solrCloudCollectionConfig/managed-schema
   [junit4]   2> 	11	/solr/configs/solrCloudCollectionConfig
   [junit4]   2> 
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2> 
   [junit4]   2> 	14	/solr/clusterstate.json
   [junit4]   2> 	12	/solr/collections/test_col/state.json
   [junit4]   2> 	2	/solr/collections/test_col/leader_elect/shard2/election/95812325951733789-core_node11-n_0000000001
   [junit4]   2> 
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2> 
   [junit4]   2> 	14	/solr/live_nodes
   [junit4]   2> 	14	/solr/collections
   [junit4]   2> 
   [junit4]   2> 3930722 INFO  (SUITE-TestTolerantUpdateProcessorRandomCloud-seed#[D87C2A1E874D0DFE]-worker) [    ] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> Apr 30, 2016 1:35:08 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=HighCompressionCompressingStoredFields(storedFieldsFormat=CompressingStoredFieldsFormat(compressionMode=HIGH_COMPRESSION, chunkSize=20452, maxDocsPerChunk=3, blockSize=31), termVectorsFormat=CompressingTermVectorsFormat(compressionMode=HIGH_COMPRESSION, chunkSize=20452, blockSize=31)), sim=ClassicSimilarity, locale=sk, timezone=Etc/GMT-7
   [junit4]   2> NOTE: Linux 3.13.0-85-generic amd64/Oracle Corporation 1.8.0_74 (64-bit)/cpus=4,threads=1,free=199675744,total=515375104
   [junit4]   2> NOTE: All tests run in this JVM: [SolrCmdDistributorTest, SolrIndexSplitterTest, TestRequestStatusCollectionAPI, TimeZoneUtilsTest, SuggestComponentTest, DirectSolrSpellCheckerTest, TestTrie, CollectionsAPIDistributedZkTest, HdfsBasicDistributedZkTest, SoftAutoCommitTest, TestQueryUtils, AddSchemaFieldsUpdateProcessorFactoryTest, TestExclusionRuleCollectionAccess, TestCustomSort, DeleteReplicaTest, SpellingQueryConverterTest, TestAnalyzeInfixSuggestions, TestJsonFacets, CSVRequestHandlerTest, TriLevelCompositeIdRoutingTest, TestDynamicFieldCollectionResource, TestLFUCache, CloudMLTQParserTest, TolerantUpdateProcessorTest, TestSolrQueryResponse, TestUseDocValuesAsStored, TestSimpleTrackingShardHandler, TestObjectReleaseTracker, TestMacros, TestTestInjection, TestSolr4Spatial2, TestCoreContainer, TestPartialUpdateDeduplication, SecurityConfHandlerTest, TestReloadAndDeleteDocs, CollectionsAPIAsyncDistributedZkTest, DistributedExpandComponentTest, NumericFieldsTest, TestDFISimilarityFactory, TestLRUStatsCache, ShardRoutingCustomTest, SolrInfoMBeanTest, TestCodecSupport, TestSchemaVersionResource, TestSolrConfigHandlerCloud, ZkStateReaderTest, BasicFunctionalityTest, SolrIndexConfigTest, CdcrUpdateLogTest, SpellCheckCollatorTest, TestConfigOverlay, AliasIntegrationTest, TestMissingGroups, CustomCollectionTest, CollectionReloadTest, TestQuerySenderNoQuery, DocValuesTest, RecoveryAfterSoftCommitTest, UniqFieldsUpdateProcessorFactoryTest, FastVectorHighlighterTest, TestCloudInspectUtil, TestSolrDeletionPolicy2, LoggingHandlerTest, SaslZkACLProviderTest, TestAuthorizationFramework, HighlighterTest, DistribCursorPagingTest, FileUtilsTest, TestSortByMinMaxFunction, TestHashPartitioner, TestManagedSynonymFilterFactory, TestHashQParserPlugin, FacetPivotSmallTest, SchemaVersionSpecificBehaviorTest, ConvertedLegacyTest, TestDFRSimilarityFactory, BlockJoinFacetRandomTest, TestBinaryResponseWriter, DateRangeFieldTest, TestFreeTextSuggestions, HighlighterMaxOffsetTest, TestWordDelimiterFilterFactory, TestBlobHandler, TestSurroundQueryParser, NotRequiredUniqueKeyTest, OutOfBoxZkACLAndCredentialsProvidersTest, TestSchemaNameResource, HdfsThreadLeakTest, DocumentBuilderTest, TestGeoJSONResponseWriter, SpatialFilterTest, AnalyticsQueryTest, TermsComponentTest, HdfsCollectionsAPIDistributedZkTest, RuleEngineTest, MoreLikeThisHandlerTest, OverseerTest, RecoveryZkTest, ZkSolrClientTest, TestZkChroot, ZkCLITest, TestDistributedSearch, TestCursorMarkWithoutUniqueKey, TestDistributedMissingSort, TestEmbeddedSolrServerConstructors, TestJettySolrRunner, AsyncMigrateRouteKeyTest, CdcrReplicationHandlerTest, CollectionTooManyReplicasTest, ConcurrentDeleteAndCreateCollectionTest, DeleteStatusTest, DistributedQueueTest, HttpPartitionTest, LeaderInitiatedRecoveryOnShardRestartTest, RemoteQueryErrorTest, ShardSplitTest, TestOnReconnectListenerSupport, TestRebalanceLeaders, TestTolerantUpdateProcessorRandomCloud]
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestTolerantUpdateProcessorRandomCloud -Dtests.seed=D87C2A1E874D0DFE -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=sk -Dtests.timezone=Etc/GMT-7 -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR   0.00s J1 | TestTolerantUpdateProcessorRandomCloud (suite) <<<
   [junit4]    > Throwable #1: java.lang.AssertionError: Could not find collection:test_col
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([D87C2A1E874D0DFE]:0)
   [junit4]    > 	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:150)
   [junit4]    > 	at org.apache.solr.cloud.TestTolerantUpdateProcessorRandomCloud.createMiniSolrCloudCluster(TestTolerantUpdateProcessorRandomCloud.java:135)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4] Completed [474/592 (3!)] on J1 in 37.89s, 0 tests, 1 failure <<< FAILURES!

[...truncated 385 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:747: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:684: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/common-build.xml:529: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/common-build.xml:1457: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/common-build.xml:1014: There were test failures: 592 suites (10 ignored), 2397 tests, 1 suite-level error, 1 error, 1 failure, 31 ignored (16 assumptions) [seed: D87C2A1E874D0DFE]

Total time: 253 minutes 11 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
No prior successful build to compare, so performing full copy of artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any




[JENKINS] Lucene-Solr-NightlyTests-6.x - Build # 58 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-6.x/58/

7 tests failed.
FAILED:  org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest.test

Error Message:
document count mismatch.  control=3941 sum(shards)=3940 cloudClient=3940

Stack Trace:
java.lang.AssertionError: document count mismatch.  control=3941 sum(shards)=3940 cloudClient=3940
	at __randomizedtesting.SeedInfo.seed([8F44261516A7A083:71019CFB85BCD7B]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1319)
	at org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest.test(ChaosMonkeyNothingIsSafeTest.java:228)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  junit.framework.TestSuite.org.apache.solr.handler.component.TermVectorComponentDistributedTest

Error Message:


Stack Trace:
java.lang.AssertionError
	at org.apache.lucene.util.TestRuleTemporaryFilesCleanup.before(TestRuleTemporaryFilesCleanup.java:112)
	at com.carrotsearch.randomizedtesting.rules.TestRuleAdapter$1.before(TestRuleAdapter.java:26)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:35)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  junit.framework.TestSuite.org.apache.solr.handler.component.TermVectorComponentDistributedTest

Error Message:
71 threads leaked from SUITE scope at org.apache.solr.handler.component.TermVectorComponentDistributedTest:     1) Thread[id=8465, name=coreZkRegister-1049-thread-18-processing-n:127.0.0.1:37006_, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.cloud.ShardLeaderElectionContext.waitForReplicasToComeUp(ElectionContext.java:616)         at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:318)         at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:170)         at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:135)         at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:307)         at org.apache.solr.cloud.ZkController.joinElection(ZkController.java:1042)         at org.apache.solr.cloud.ZkController.register(ZkController.java:853)         at org.apache.solr.cloud.ZkController$RegisterCoreAsync.call(ZkController.java:203)         at java.util.concurrent.FutureTask.run(FutureTask.java:266)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$8/1049045488.run(Unknown Source)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)    2) Thread[id=8523, name=Async disk worker #2 for volume /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_8F44261516A7A083-001/tempDir-001/hdfsBaseDir/data/data3/current, state=TIMED_WAITING, group=FsDatasetAsyncDiskService]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)    3) Thread[id=8524, name=Async disk worker #2 for volume /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_8F44261516A7A083-001/tempDir-001/hdfsBaseDir/data/data1/current, state=TIMED_WAITING, group=FsDatasetAsyncDiskService]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)    4) Thread[id=8451, name=coreZkRegister-1049-thread-4-processing-n:127.0.0.1:37006_, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.cloud.ShardLeaderElectionContext.waitForReplicasToComeUp(ElectionContext.java:616)         at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:318)         at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:170)         at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:135)         at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:307)         at org.apache.solr.cloud.ZkController.joinElection(ZkController.java:1042)         at org.apache.solr.cloud.ZkController.register(ZkController.java:853)         at org.apache.solr.cloud.ZkController$RegisterCoreAsync.call(ZkController.java:203)         at java.util.concurrent.FutureTask.run(FutureTask.java:266)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$8/1049045488.run(Unknown Source)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)    5) Thread[id=8468, name=coreZkRegister-1049-thread-21-processing-n:127.0.0.1:37006_, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.cloud.ShardLeaderElectionContext.waitForReplicasToComeUp(ElectionContext.java:616)         at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:318)         at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:170)         at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:135)         at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:307)         at org.apache.solr.cloud.ZkController.joinElection(ZkController.java:1042)         at org.apache.solr.cloud.ZkController.register(ZkController.java:853)         at org.apache.solr.cloud.ZkController$RegisterCoreAsync.call(ZkController.java:203)         at java.util.concurrent.FutureTask.run(FutureTask.java:266)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$8/1049045488.run(Unknown Source)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)    6) Thread[id=8498, name=coreZkRegister-1208-thread-19, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)    7) Thread[id=8512, name=updateExecutor-493-thread-6-processing-n:127.0.0.1:50321_ x:awholynewstresscollection_collection4_0_shard1_replica4 s:shard1 c:awholynewstresscollection_collection4_0 r:core_node6, state=WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.FutureTask.awaitDone(FutureTask.java:429)         at java.util.concurrent.FutureTask.get(FutureTask.java:191)         at org.apache.solr.update.DefaultSolrCoreState$2.run(DefaultSolrCoreState.java:317)         at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)         at java.util.concurrent.FutureTask.run(FutureTask.java:266)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$8/1049045488.run(Unknown Source)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)    8) Thread[id=8172, name=java.util.concurrent.ThreadPoolExecutor$Worker@3c2734e6[State = -1, empty queue], state=WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)    9) Thread[id=8475, name=OverseerThreadFactory-1932-thread-4, state=WAITING, group=Overseer collection creation process.]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   10) Thread[id=8489, name=coreZkRegister-1208-thread-10, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   11) Thread[id=8457, name=coreZkRegister-1049-thread-10, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   12) Thread[id=8494, name=coreZkRegister-1208-thread-15-processing-n:127.0.0.1:50321_, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.cloud.ZkController.getLeaderProps(ZkController.java:1000)         at org.apache.solr.cloud.ZkController.getLeaderProps(ZkController.java:970)         at org.apache.solr.cloud.ZkController.getLeader(ZkController.java:926)         at org.apache.solr.cloud.ZkController.register(ZkController.java:865)         at org.apache.solr.cloud.ZkController$RegisterCoreAsync.call(ZkController.java:203)         at java.util.concurrent.FutureTask.run(FutureTask.java:266)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$8/1049045488.run(Unknown Source)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   13) Thread[id=8438, name=qtp1292668453-8438, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:546)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:609)         at java.lang.Thread.run(Thread.java:745)   14) Thread[id=8493, name=coreZkRegister-1208-thread-14-processing-n:127.0.0.1:50321_, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.cloud.ZkController.getLeaderProps(ZkController.java:1000)         at org.apache.solr.cloud.ZkController.getLeaderProps(ZkController.java:970)         at org.apache.solr.cloud.ZkController.getLeader(ZkController.java:926)         at org.apache.solr.cloud.ZkController.register(ZkController.java:865)         at org.apache.solr.cloud.ZkController$RegisterCoreAsync.call(ZkController.java:203)         at java.util.concurrent.FutureTask.run(FutureTask.java:266)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$8/1049045488.run(Unknown Source)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   15) Thread[id=8491, name=coreZkRegister-1208-thread-12-processing-n:127.0.0.1:50321_, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.cloud.ZkController.getLeaderProps(ZkController.java:1000)         at org.apache.solr.cloud.ZkController.getLeaderProps(ZkController.java:970)         at org.apache.solr.cloud.ZkController.getLeader(ZkController.java:926)         at org.apache.solr.cloud.ZkController.register(ZkController.java:865)         at org.apache.solr.cloud.ZkController$RegisterCoreAsync.call(ZkController.java:203)         at java.util.concurrent.FutureTask.run(FutureTask.java:266)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$8/1049045488.run(Unknown Source)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   16) Thread[id=8462, name=coreZkRegister-1049-thread-15-processing-n:127.0.0.1:37006_, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.cloud.ShardLeaderElectionContext.waitForReplicasToComeUp(ElectionContext.java:616)         at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:318)         at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:170)         at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:135)         at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:307)         at org.apache.solr.cloud.ZkController.joinElection(ZkController.java:1042)         at org.apache.solr.cloud.ZkController.register(ZkController.java:853)         at org.apache.solr.cloud.ZkController$RegisterCoreAsync.call(ZkController.java:203)         at java.util.concurrent.FutureTask.run(FutureTask.java:266)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$8/1049045488.run(Unknown Source)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   17) Thread[id=8418, name=IPC Parameter Sending Thread #38, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   18) Thread[id=8499, name=coreZkRegister-1208-thread-20, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   19) Thread[id=8520, name=zkCallback-460-thread-17, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   20) Thread[id=8455, name=coreZkRegister-1049-thread-8, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   21) Thread[id=8488, name=coreZkRegister-1208-thread-9-processing-n:127.0.0.1:50321_, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.cloud.ShardLeaderElectionContext.waitForReplicasToComeUp(ElectionContext.java:616)         at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:318)         at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:170)         at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:135)         at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:307)         at org.apache.solr.cloud.ZkController.joinElection(ZkController.java:1042)         at org.apache.solr.cloud.ZkController.register(ZkController.java:853)         at org.apache.solr.cloud.ZkController$RegisterCoreAsync.call(ZkController.java:203)         at java.util.concurrent.FutureTask.run(FutureTask.java:266)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$8/1049045488.run(Unknown Source)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   22) Thread[id=8477, name=OverseerThreadFactory-1932-thread-6, state=WAITING, group=Overseer collection creation process.]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   23) Thread[id=8444, name=OverseerStateUpdate-95850796072239334-127.0.0.1:50321_-n_0000000006, state=TIMED_WAITING, group=Overseer state updater.]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.apache.solr.cloud.DistributedQueue.peek(DistributedQueue.java:168)         at org.apache.solr.cloud.DistributedQueue.peek(DistributedQueue.java:142)         at org.apache.solr.cloud.Overseer$ClusterStateUpdater.run(Overseer.java:196)         at java.lang.Thread.run(Thread.java:745)   24) Thread[id=8452, name=coreZkRegister-1049-thread-5-processing-n:127.0.0.1:37006_, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.cloud.ShardLeaderElectionContext.waitForReplicasToComeUp(ElectionContext.java:616)         at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:318)         at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:170)         at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:135)         at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:307)         at org.apache.solr.cloud.ZkController.joinElection(ZkController.java:1042)         at org.apache.solr.cloud.ZkController.register(ZkController.java:853)         at org.apache.solr.cloud.ZkController$RegisterCoreAsync.call(ZkController.java:203)         at java.util.concurrent.FutureTask.run(FutureTask.java:266)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$8/1049045488.run(Unknown Source)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   25) Thread[id=8486, name=coreZkRegister-1208-thread-7-processing-n:127.0.0.1:50321_, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.cloud.ShardLeaderElectionContext.waitForReplicasToComeUp(ElectionContext.java:616)         at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:318)         at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:170)         at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:135)         at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:307)         at org.apache.solr.cloud.ZkController.joinElection(ZkController.java:1042)         at org.apache.solr.cloud.ZkController.register(ZkController.java:853)         at org.apache.solr.cloud.ZkController$RegisterCoreAsync.call(ZkController.java:203)         at java.util.concurrent.FutureTask.run(FutureTask.java:266)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$8/1049045488.run(Unknown Source)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   26) Thread[id=8443, name=qtp1208954681-8443, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:546)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:609)         at java.lang.Thread.run(Thread.java:745)   27) Thread[id=8449, name=zkCallback-465-thread-12, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   28) Thread[id=8476, name=OverseerThreadFactory-1932-thread-5, state=WAITING, group=Overseer collection creation process.]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   29) Thread[id=8514, name=recoveryExecutor-494-thread-6-processing-n:127.0.0.1:50321_ x:awholynewstresscollection_collection4_0_shard1_replica4 s:shard1 c:awholynewstresscollection_collection4_0 r:core_node6, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.common.cloud.ZkStateReader.getLeaderRetry(ZkStateReader.java:675)         at org.apache.solr.common.cloud.ZkStateReader.getLeaderRetry(ZkStateReader.java:663)         at org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:305)         at org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:221)         at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)         at java.util.concurrent.FutureTask.run(FutureTask.java:266)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$8/1049045488.run(Unknown Source)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   30) Thread[id=8484, name=coreZkRegister-1208-thread-5-processing-n:127.0.0.1:50321_, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.cloud.ZkController.getLeaderProps(ZkController.java:1000)         at org.apache.solr.cloud.ZkController.getLeaderProps(ZkController.java:970)         at org.apache.solr.cloud.ZkController.getLeader(ZkController.java:926)         at org.apache.solr.cloud.ZkController.register(ZkController.java:865)         at org.apache.solr.cloud.ZkController$RegisterCoreAsync.call(ZkController.java:203)         at java.util.concurrent.FutureTask.run(FutureTask.java:266)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$8/1049045488.run(Unknown Source)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   31) Thread[id=8474, name=OverseerThreadFactory-1932-thread-3, state=WAITING, group=Overseer collection creation process.]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   32) Thread[id=8471, name=coreZkRegister-1049-thread-23, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   33) Thread[id=8415, name=IPC Client (2081512138) connection to localhost/127.0.0.1:53213 from jenkins, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at java.lang.Object.wait(Native Method)         at org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:920)         at org.apache.hadoop.ipc.Client$Connection.run(Client.java:965)   34) Thread[id=8473, name=OverseerThreadFactory-1932-thread-2, state=WAITING, group=Overseer collection creation process.]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   35) Thread[id=8464, name=coreZkRegister-1049-thread-17-processing-n:127.0.0.1:37006_, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.cloud.ShardLeaderElectionContext.waitForReplicasToComeUp(ElectionContext.java:616)         at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:318)         at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:170)         at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:135)         at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:307)         at org.apache.solr.cloud.ZkController.joinElection(ZkController.java:1042)         at org.apache.solr.cloud.ZkController.register(ZkController.java:853)         at org.apache.solr.cloud.ZkController$RegisterCoreAsync.call(ZkController.java:203)         at java.util.concurrent.FutureTask.run(FutureTask.java:266)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$8/1049045488.run(Unknown Source)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   36) Thread[id=8496, name=coreZkRegister-1208-thread-17-processing-n:127.0.0.1:50321_, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.cloud.ShardLeaderElectionContext.waitForReplicasToComeUp(ElectionContext.java:616)         at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:318)         at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:170)         at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:135)         at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:307)         at org.apache.solr.cloud.ZkController.joinElection(ZkController.java:1042)         at org.apache.solr.cloud.ZkController.register(ZkController.java:853)         at org.apache.solr.cloud.ZkController$RegisterCoreAsync.call(ZkController.java:203)         at java.util.concurrent.FutureTask.run(FutureTask.java:266)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$8/1049045488.run(Unknown Source)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   37) Thread[id=8490, name=coreZkRegister-1208-thread-11, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   38) Thread[id=8417, name=IPC Parameter Sending Thread #37, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   39) Thread[id=8485, name=coreZkRegister-1208-thread-6, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   40) Thread[id=8469, name=coreZkRegister-1049-thread-22-processing-n:127.0.0.1:37006_, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.cloud.ShardLeaderElectionContext.waitForReplicasToComeUp(ElectionContext.java:616)         at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:318)         at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:170)         at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:135)         at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:307)         at org.apache.solr.cloud.ZkController.joinElection(ZkController.java:1042)         at org.apache.solr.cloud.ZkController.register(ZkController.java:853)         at org.apache.solr.cloud.ZkController$RegisterCoreAsync.call(ZkController.java:203)         at java.util.concurrent.FutureTask.run(FutureTask.java:266)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$8/1049045488.run(Unknown Source)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   41) Thread[id=8441, name=qtp460475988-8441, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:233)         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:181)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:171)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:156)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)         at java.lang.Thread.run(Thread.java:745)   42) Thread[id=8447, name=zkCallback-496-thread-20, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   43) Thread[id=8448, name=coreZkRegister-1049-thread-2-processing-n:127.0.0.1:37006_, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.cloud.ShardLeaderElectionContext.waitForReplicasToComeUp(ElectionContext.java:616)         at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:318)         at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:170)         at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:135)         at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:307)         at org.apache.solr.cloud.ZkController.joinElection(ZkController.java:1042)         at org.apache.solr.cloud.ZkController.register(ZkController.java:853)         at org.apache.solr.cloud.ZkController$RegisterCoreAsync.call(ZkController.java:203)         at java.util.concurrent.FutureTask.run(FutureTask.java:266)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$8/1049045488.run(Unknown Source)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   44) Thread[id=8459, name=coreZkRegister-1049-thread-12, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   45) Thread[id=8497, name=coreZkRegister-1208-thread-18-processing-n:127.0.0.1:50321_, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.cloud.ShardLeaderElectionContext.waitForReplicasToComeUp(ElectionContext.java:616)         at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:318)         at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:170)         at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:135)         at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:307)         at org.apache.solr.cloud.ZkController.joinElection(ZkController.java:1042)         at org.apache.solr.cloud.ZkController.register(ZkController.java:853)         at org.apache.solr.cloud.ZkController$RegisterCoreAsync.call(ZkController.java:203)         at java.util.concurrent.FutureTask.run(FutureTask.java:266)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$8/1049045488.run(Unknown Source)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   46) Thread[id=8461, name=coreZkRegister-1049-thread-14-processing-n:127.0.0.1:37006_, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.cloud.ShardLeaderElectionContext.waitForReplicasToComeUp(ElectionContext.java:616)         at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:318)         at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:170)         at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:135)         at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:307)         at org.apache.solr.cloud.ZkController.joinElection(ZkController.java:1042)         at org.apache.solr.cloud.ZkController.register(ZkController.java:853)         at org.apache.solr.cloud.ZkController$RegisterCoreAsync.call(ZkController.java:203)         at java.util.concurrent.FutureTask.run(FutureTask.java:266)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$8/1049045488.run(Unknown Source)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   47) Thread[id=8467, name=coreZkRegister-1049-thread-20-processing-n:127.0.0.1:37006_, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.cloud.ShardLeaderElectionContext.waitForReplicasToComeUp(ElectionContext.java:616)         at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:318)         at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:170)         at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:135)         at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:307)         at org.apache.solr.cloud.ZkController.joinElection(ZkController.java:1042)         at org.apache.solr.cloud.ZkController.register(ZkController.java:853)         at org.apache.solr.cloud.ZkController$RegisterCoreAsync.call(ZkController.java:203)         at java.util.concurrent.FutureTask.run(FutureTask.java:266)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$8/1049045488.run(Unknown Source)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   48) Thread[id=8435, name=qtp199004788-8435, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:546)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:609)         at java.lang.Thread.run(Thread.java:745)   49) Thread[id=8445, name=OverseerCollectionConfigSetProcessor-95850796072239334-127.0.0.1:50321_-n_0000000006, state=TIMED_WAITING, group=Overseer collection creation process.]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.apache.solr.cloud.DistributedQueue.getChildren(DistributedQueue.java:342)         at org.apache.solr.cloud.OverseerTaskQueue.peekTopN(OverseerTaskQueue.java:235)         at org.apache.solr.cloud.OverseerTaskProcessor.run(OverseerTaskProcessor.java:200)         at java.lang.Thread.run(Thread.java:745)   50) Thread[id=8442, name=qtp1208954681-8442, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:233)         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:181)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:171)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:156)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)         at java.lang.Thread.run(Thread.java:745)   51) Thread[id=8437, name=qtp1292668453-8437, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:546)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:609)         at java.lang.Thread.run(Thread.java:745)   52) Thread[id=8522, name=Async disk worker #2 for volume /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_8F44261516A7A083-001/tempDir-001/hdfsBaseDir/data/data4/current, state=TIMED_WAITING, group=FsDatasetAsyncDiskService]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   53) Thread[id=8463, name=coreZkRegister-1049-thread-16-processing-n:127.0.0.1:37006_, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.cloud.ShardLeaderElectionContext.waitForReplicasToComeUp(ElectionContext.java:616)         at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:318)         at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:170)         at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:135)         at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:307)         at org.apache.solr.cloud.ZkController.joinElection(ZkController.java:1042)         at org.apache.solr.cloud.ZkController.register(ZkController.java:853)         at org.apache.solr.cloud.ZkController$RegisterCoreAsync.call(ZkController.java:203)         at java.util.concurrent.FutureTask.run(FutureTask.java:266)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$8/1049045488.run(Unknown Source)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   54) Thread[id=8466, name=coreZkRegister-1049-thread-19, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   55) Thread[id=8521, name=zkCallback-455-thread-15, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   56) Thread[id=8458, name=coreZkRegister-1049-thread-11, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   57) Thread[id=8453, name=coreZkRegister-1049-thread-6-processing-n:127.0.0.1:37006_, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.cloud.ShardLeaderElectionContext.waitForReplicasToComeUp(ElectionContext.java:616)         at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:318)         at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:170)         at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:135)         at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:307)         at org.apache.solr.cloud.ZkController.joinElection(ZkController.java:1042)         at org.apache.solr.cloud.ZkController.register(ZkController.java:853)         at org.apache.solr.cloud.ZkController$RegisterCoreAsync.call(ZkController.java:203)         at java.util.concurrent.FutureTask.run(FutureTask.java:266)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$8/1049045488.run(Unknown Source)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   58) Thread[id=8454, name=coreZkRegister-1049-thread-7, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   59) Thread[id=8487, name=coreZkRegister-1208-thread-8-processing-n:127.0.0.1:50321_, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.cloud.ShardLeaderElectionContext.waitForReplicasToComeUp(ElectionContext.java:616)         at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:318)         at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:170)         at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:135)         at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:307)         at org.apache.solr.cloud.ZkController.joinElection(ZkController.java:1042)         at org.apache.solr.cloud.ZkController.register(ZkController.java:853)         at org.apache.solr.cloud.ZkController$RegisterCoreAsync.call(ZkController.java:203)         at java.util.concurrent.FutureTask.run(FutureTask.java:266)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$8/1049045488.run(Unknown Source)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   60) Thread[id=8450, name=coreZkRegister-1049-thread-3, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   61) Thread[id=8495, name=coreZkRegister-1208-thread-16-processing-n:127.0.0.1:50321_, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.cloud.ZkController.getLeaderProps(ZkController.java:1000)         at org.apache.solr.cloud.ZkController.getLeaderProps(ZkController.java:970)         at org.apache.solr.cloud.ZkController.getLeader(ZkController.java:926)         at org.apache.solr.cloud.ZkController.register(ZkController.java:865)         at org.apache.solr.cloud.ZkController$RegisterCoreAsync.call(ZkController.java:203)         at java.util.concurrent.FutureTask.run(FutureTask.java:266)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$8/1049045488.run(Unknown Source)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   62) Thread[id=8492, name=coreZkRegister-1208-thread-13-processing-n:127.0.0.1:50321_, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.cloud.ZkController.getLeaderProps(ZkController.java:1000)         at org.apache.solr.cloud.ZkController.getLeaderProps(ZkController.java:970)         at org.apache.solr.cloud.ZkController.getLeader(ZkController.java:926)         at org.apache.solr.cloud.ZkController.register(ZkController.java:865)         at org.apache.solr.cloud.ZkController$RegisterCoreAsync.call(ZkController.java:203)         at java.util.concurrent.FutureTask.run(FutureTask.java:266)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$8/1049045488.run(Unknown Source)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   63) Thread[id=8158, name=IPC Client (2081512138) connection to localhost/127.0.0.1:53213 from jenkins, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335)         at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157)         at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161)         at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131)         at java.io.FilterInputStream.read(FilterInputStream.java:133)         at java.io.FilterInputStream.read(FilterInputStream.java:133)         at org.apache.hadoop.ipc.Client$Connection$PingInputStream.read(Client.java:513)         at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)         at java.io.BufferedInputStream.read(BufferedInputStream.java:265)         at java.io.DataInputStream.readInt(DataInputStream.java:387)         at org.apache.hadoop.ipc.Client$Connection.receiveRpcResponse(Client.java:1071)         at org.apache.hadoop.ipc.Client$Connection.run(Client.java:966)   64) Thread[id=8470, name=zkCallback-465-thread-13, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   65) Thread[id=8483, name=coreZkRegister-1208-thread-4, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   66) Thread[id=8446, name=OverseerHdfsCoreFailoverThread-95850796072239334-127.0.0.1:50321_-n_0000000006, state=TIMED_WAITING, group=Overseer Hdfs SolrCore Failover Thread.]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.cloud.OverseerAutoReplicaFailoverThread.run(OverseerAutoReplicaFailoverThread.java:138)         at java.lang.Thread.run(Thread.java:745)   67) Thread[id=8516, name=updateExecutor-493-thread-7, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   68) Thread[id=8460, name=coreZkRegister-1049-thread-13, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   69) Thread[id=8456, name=coreZkRegister-1049-thread-9, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   70) Thread[id=8518, name=recoveryExecutor-494-thread-7, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   71) Thread[id=8525, name=Async disk worker #2 for volume /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_8F44261516A7A083-001/tempDir-001/hdfsBaseDir/data/data2/current, state=TIMED_WAITING, group=FsDatasetAsyncDiskService]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 71 threads leaked from SUITE scope at org.apache.solr.handler.component.TermVectorComponentDistributedTest: 
   1) Thread[id=8465, name=coreZkRegister-1049-thread-18-processing-n:127.0.0.1:37006_, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]
        at java.lang.Thread.sleep(Native Method)
        at org.apache.solr.cloud.ShardLeaderElectionContext.waitForReplicasToComeUp(ElectionContext.java:616)
        at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:318)
        at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:170)
        at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:135)
        at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:307)
        at org.apache.solr.cloud.ZkController.joinElection(ZkController.java:1042)
        at org.apache.solr.cloud.ZkController.register(ZkController.java:853)
        at org.apache.solr.cloud.ZkController$RegisterCoreAsync.call(ZkController.java:203)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
        at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$8/1049045488.run(Unknown Source)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
   2) Thread[id=8523, name=Async disk worker #2 for volume /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_8F44261516A7A083-001/tempDir-001/hdfsBaseDir/data/data3/current, state=TIMED_WAITING, group=FsDatasetAsyncDiskService]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
   3) Thread[id=8524, name=Async disk worker #2 for volume /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_8F44261516A7A083-001/tempDir-001/hdfsBaseDir/data/data1/current, state=TIMED_WAITING, group=FsDatasetAsyncDiskService]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
   4) Thread[id=8451, name=coreZkRegister-1049-thread-4-processing-n:127.0.0.1:37006_, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]
        at java.lang.Thread.sleep(Native Method)
        at org.apache.solr.cloud.ShardLeaderElectionContext.waitForReplicasToComeUp(ElectionContext.java:616)
        at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:318)
        at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:170)
        at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:135)
        at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:307)
        at org.apache.solr.cloud.ZkController.joinElection(ZkController.java:1042)
        at org.apache.solr.cloud.ZkController.register(ZkController.java:853)
        at org.apache.solr.cloud.ZkController$RegisterCoreAsync.call(ZkController.java:203)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
        at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$8/1049045488.run(Unknown Source)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
   5) Thread[id=8468, name=coreZkRegister-1049-thread-21-processing-n:127.0.0.1:37006_, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]
        at java.lang.Thread.sleep(Native Method)
        at org.apache.solr.cloud.ShardLeaderElectionContext.waitForReplicasToComeUp(ElectionContext.java:616)
        at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:318)
        at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:170)
        at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:135)
        at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:307)
        at org.apache.solr.cloud.ZkController.joinElection(ZkController.java:1042)
        at org.apache.solr.cloud.ZkController.register(ZkController.java:853)
        at org.apache.solr.cloud.ZkController$RegisterCoreAsync.call(ZkController.java:203)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
        at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$8/1049045488.run(Unknown Source)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
   6) Thread[id=8498, name=coreZkRegister-1208-thread-19, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
   7) Thread[id=8512, name=updateExecutor-493-thread-6-processing-n:127.0.0.1:50321_ x:awholynewstresscollection_collection4_0_shard1_replica4 s:shard1 c:awholynewstresscollection_collection4_0 r:core_node6, state=WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.FutureTask.awaitDone(FutureTask.java:429)
        at java.util.concurrent.FutureTask.get(FutureTask.java:191)
        at org.apache.solr.update.DefaultSolrCoreState$2.run(DefaultSolrCoreState.java:317)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
        at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$8/1049045488.run(Unknown Source)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
   8) Thread[id=8172, name=java.util.concurrent.ThreadPoolExecutor$Worker@3c2734e6[State = -1, empty queue], state=WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
   9) Threa

[...truncated too long message...]

_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
   [junit4]    >         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >   52) Thread[id=8499, name=coreZkRegister-1208-thread-20, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
   [junit4]    >         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >   53) Thread[id=8458, name=coreZkRegister-1049-thread-11, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
   [junit4]    >         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >   54) Thread[id=8453, name=coreZkRegister-1049-thread-6, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
   [junit4]    >         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >   55) Thread[id=8464, name=coreZkRegister-1049-thread-17, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
   [junit4]    >         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >   56) Thread[id=8518, name=recoveryExecutor-494-thread-7, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
   [junit4]    >         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >   57) Thread[id=8525, name=Async disk worker #2 for volume /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_8F44261516A7A083-001/tempDir-001/hdfsBaseDir/data/data2/current, state=TIMED_WAITING, group=FsDatasetAsyncDiskService]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
   [junit4]    >         at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([8F44261516A7A083]:0)Throwable #4: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=2833, name=Thread-1066, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]
   [junit4]    > Caused by: java.lang.OutOfMemoryError: Java heap spaceThrowable #5: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=2753, name=IPC Server idle connection scanner for port 53213, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]
   [junit4]    > Caused by: java.lang.OutOfMemoryError: Java heap space
   [junit4] Completed [519/595 (4!)] on J2 in 8163.44s, 0 tests, 1 failure, 4 errors <<< FAILURES!

[...truncated 229 lines...]
   [junit4] JVM J2: stdout was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/temp/junit4-J2-20160506_203053_666.sysout
   [junit4] >>> JVM J2 emitted unexpected output (verbatim) ----
   [junit4] java.lang.OutOfMemoryError: Java heap space
   [junit4] Dumping heap to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/heapdumps/java_pid10577.hprof ...
   [junit4] Heap dump file created [682066696 bytes in 5.571 secs]
   [junit4] <<< JVM J2: EOF ----

[...truncated 13 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:747: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:684: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/common-build.xml:529: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/common-build.xml:1457: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/common-build.xml:1014: There were test failures: 595 suites (10 ignored), 2403 tests, 6 suite-level errors, 1 error, 1 failure, 361 ignored (13 assumptions) [seed: 8F44261516A7A083]

Total time: 558 minutes 35 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
No prior successful build to compare, so performing full copy of artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any




[JENKINS] Lucene-Solr-NightlyTests-6.x - Build # 57 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-6.x/57/

4 tests failed.
FAILED:  org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.test

Error Message:
Timeout occured while waiting response from server at: http://127.0.0.1:58336

Stack Trace:
org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:58336
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:601)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:259)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:248)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.makeRequest(CollectionsAPIDistributedZkTest.java:399)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testErrorHandling(CollectionsAPIDistributedZkTest.java:457)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test(CollectionsAPIDistributedZkTest.java:179)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.net.SocketTimeoutException: Read timed out
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
	at java.net.SocketInputStream.read(SocketInputStream.java:170)
	at java.net.SocketInputStream.read(SocketInputStream.java:141)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:160)
	at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:84)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:273)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
	at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
	at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)
	at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)
	at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197)
	at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)
	at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)
	at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685)
	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487)
	at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:495)
	... 47 more


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest

Error Message:
1 thread leaked from SUITE scope at org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest:     1) Thread[id=7672, name=searcherExecutor-2383-thread-1, state=WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest: 
   1) Thread[id=7672, name=searcherExecutor-2383-thread-1, state=WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
	at __randomizedtesting.SeedInfo.seed([E363401D1F6FC7BD]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest

Error Message:
There are still zombie threads that couldn't be terminated:    1) Thread[id=7672, name=searcherExecutor-2383-thread-1, state=WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   1) Thread[id=7672, name=searcherExecutor-2383-thread-1, state=WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
	at __randomizedtesting.SeedInfo.seed([E363401D1F6FC7BD]:0)


FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=115839, name=collection0, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=115839, name=collection0, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:59048: Could not find collection : awholynewstresscollection_collection0_2
	at __randomizedtesting.SeedInfo.seed([E363401D1F6FC7BD]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:590)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:259)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:248)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:404)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:357)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1192)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:962)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:898)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:994)




Build Log:
[...truncated 11450 lines...]
   [junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_E363401D1F6FC7BD-001/init-core-data-001
   [junit4]   2> 1250651 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[E363401D1F6FC7BD]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false)
   [junit4]   2> 1250651 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[E363401D1F6FC7BD]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   2> 1250656 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 1250678 INFO  (Thread-3866) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1250678 INFO  (Thread-3866) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 1250770 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.c.ZkTestServer start zk server on port:54018
   [junit4]   2> 1250770 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1250794 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1250802 INFO  (zkCallback-30613-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@228fb82b name:ZooKeeperConnection Watcher:127.0.0.1:54018 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1250802 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1250802 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1250802 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 1250814 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1250826 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1250842 INFO  (zkCallback-30614-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6d376033 name:ZooKeeperConnection Watcher:127.0.0.1:54018/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1250842 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1250843 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1250843 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 1250843 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 1250844 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 1250844 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 1250845 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 1250845 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 1250845 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 1250846 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 1250846 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1250846 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1250847 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 1250847 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 1250847 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 1250848 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 1250848 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 1250848 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 1250849 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 1250849 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 1250849 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1250849 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1250850 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1250850 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1250850 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 1250850 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 1250851 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 1250851 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 1250852 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1250866 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1250879 INFO  (zkCallback-30615-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@29181fe7 name:ZooKeeperConnection Watcher:127.0.0.1:54018 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1250879 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1250879 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1250879 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 1250880 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1250880 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1250881 INFO  (zkCallback-30616-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4db05ded name:ZooKeeperConnection Watcher:127.0.0.1:54018/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1250881 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1250881 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1250881 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf2/solrconfig.xml
   [junit4]   2> 1250881 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/solrconfig.xml
   [junit4]   2> 1250882 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf2/schema.xml
   [junit4]   2> 1250883 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/schema.xml
   [junit4]   2> 1250883 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf2/enumsConfig.xml
   [junit4]   2> 1250883 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/enumsConfig.xml
   [junit4]   2> 1250884 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf2/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1250884 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1250885 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf2/stopwords.txt
   [junit4]   2> 1250885 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/stopwords.txt
   [junit4]   2> 1250886 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf2/protwords.txt
   [junit4]   2> 1250886 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/protwords.txt
   [junit4]   2> 1250886 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf2/currency.xml
   [junit4]   2> 1250887 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/currency.xml
   [junit4]   2> 1250887 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf2/open-exchange-rates.json
   [junit4]   2> 1250887 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/open-exchange-rates.json
   [junit4]   2> 1250888 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf2/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1250888 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1250889 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf2/old_synonyms.txt
   [junit4]   2> 1250889 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/old_synonyms.txt
   [junit4]   2> 1250889 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf2/synonyms.txt
   [junit4]   2> 1250890 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/synonyms.txt
   [junit4]   2> 1250890 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/elevate.xml to /configs/conf2/elevate.xml
   [junit4]   2> 1250890 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/elevate.xml
   [junit4]   2> 1250947 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_E363401D1F6FC7BD-001/control-001/cores/collection1
   [junit4]   2> 1250948 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.e.j.s.Server jetty-9.3.8.v20160314
   [junit4]   2> 1250949 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1fbb6727{/,null,AVAILABLE}
   [junit4]   2> 1250974 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.e.j.s.ServerConnector Started ServerConnector@34c8d22{HTTP/1.1,[http/1.1]}{127.0.0.1:33749}
   [junit4]   2> 1250974 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.e.j.s.Server Started @1253105ms
   [junit4]   2> 1250974 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_E363401D1F6FC7BD-001/tempDir-001/control/data, hostContext=/, hostPort=33749, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_E363401D1F6FC7BD-001/control-001/cores}
   [junit4]   2> 1250975 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@73d16e93
   [junit4]   2> 1250975 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_E363401D1F6FC7BD-001/control-001'
   [junit4]   2> 1250975 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1250975 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1250975 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1250980 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1250990 INFO  (zkCallback-30617-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@269a429d name:ZooKeeperConnection Watcher:127.0.0.1:54018/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1250990 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1250991 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1250991 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1250991 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_E363401D1F6FC7BD-001/control-001/solr.xml
   [junit4]   2> 1250995 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_E363401D1F6FC7BD-001/control-001/cores
   [junit4]   2> 1250995 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.c.CoreContainer New CoreContainer 838482464
   [junit4]   2> 1250995 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_E363401D1F6FC7BD-001/control-001]
   [junit4]   2> 1250995 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_E363401D1F6FC7BD-001/control-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_E363401D1F6FC7BD-001/control-001/lib
   [junit4]   2> 1250995 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1250996 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1250996 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1250996 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1250996 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:54018/solr
   [junit4]   2> 1250996 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1250996 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1251006 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1251012 INFO  (zkCallback-30620-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2c8961e2 name:ZooKeeperConnection Watcher:127.0.0.1:54018 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1251012 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1251013 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1251014 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [n:127.0.0.1:33749_    ] o.a.s.c.ZkController Added new OnReconnect listener org.apache.solr.cloud.ZkController$$Lambda$10/1553166551@2eaeef44
   [junit4]   2> 1251017 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [n:127.0.0.1:33749_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1251023 INFO  (zkCallback-30621-thread-1-processing-n:127.0.0.1:33749_) [n:127.0.0.1:33749_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@661db907 name:ZooKeeperConnection Watcher:127.0.0.1:54018/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1251023 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [n:127.0.0.1:33749_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1251024 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [n:127.0.0.1:33749_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 1251024 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [n:127.0.0.1:33749_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 1251025 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [n:127.0.0.1:33749_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 1251026 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [n:127.0.0.1:33749_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 1251026 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [n:127.0.0.1:33749_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 1251027 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [n:127.0.0.1:33749_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 1251027 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [n:127.0.0.1:33749_    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 1251028 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [n:127.0.0.1:33749_    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 1251028 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [n:127.0.0.1:33749_    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 1251028 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [n:127.0.0.1:33749_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1251029 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [n:127.0.0.1:33749_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (0)
   [junit4]   2> 1251029 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [n:127.0.0.1:33749_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 1251029 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [n:127.0.0.1:33749_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 1251030 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [n:127.0.0.1:33749_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1251030 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [n:127.0.0.1:33749_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/95844247090429958-127.0.0.1:33749_-n_0000000000
   [junit4]   2> 1251030 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [n:127.0.0.1:33749_    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:33749_
   [junit4]   2> 1251030 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [n:127.0.0.1:33749_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 1251030 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [n:127.0.0.1:33749_    ] o.a.s.c.Overseer Overseer (id=95844247090429958-127.0.0.1:33749_-n_0000000000) starting
   [junit4]   2> 1251031 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [n:127.0.0.1:33749_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 1251033 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [n:127.0.0.1:33749_    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 1251034 INFO  (OverseerStateUpdate-95844247090429958-127.0.0.1:33749_-n_0000000000) [n:127.0.0.1:33749_    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 1251036 INFO  (OverseerCollectionConfigSetProcessor-95844247090429958-127.0.0.1:33749_-n_0000000000) [n:127.0.0.1:33749_    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 1251039 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [n:127.0.0.1:33749_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:33749_
   [junit4]   2> 1251039 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [n:127.0.0.1:33749_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:33749_
   [junit4]   2> 1251040 INFO  (OverseerStateUpdate-95844247090429958-127.0.0.1:33749_-n_0000000000) [n:127.0.0.1:33749_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (0)
   [junit4]   2> 1251040 INFO  (zkCallback-30621-thread-1-processing-n:127.0.0.1:33749_) [n:127.0.0.1:33749_    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [0])
   [junit4]   2> 1251040 INFO  (zkCallback-30621-thread-1-processing-n:127.0.0.1:33749_) [n:127.0.0.1:33749_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1251047 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [n:127.0.0.1:33749_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1251047 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [n:127.0.0.1:33749_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1251048 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [n:127.0.0.1:33749_    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_E363401D1F6FC7BD-001/control-001/cores
   [junit4]   2> 1251048 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [n:127.0.0.1:33749_    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, loadOnStartup=true, configSetProperties=configsetprops.json, collection=control_collection, coreNodeName=, dataDir=data/, shard=}
   [junit4]   2> 1251048 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [n:127.0.0.1:33749_    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_E363401D1F6FC7BD-001/control-001/cores/collection1
   [junit4]   2> 1251048 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [n:127.0.0.1:33749_    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 1251051 INFO  (coreLoadExecutor-7105-thread-1-processing-n:127.0.0.1:33749_) [n:127.0.0.1:33749_ c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 1251051 INFO  (coreLoadExecutor-7105-thread-1-processing-n:127.0.0.1:33749_) [n:127.0.0.1:33749_ c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1251051 INFO  (coreLoadExecutor-7105-thread-1-processing-n:127.0.0.1:33749_) [n:127.0.0.1:33749_ c:control_collection   x:collection1] o.a.s.c.ZkController Collection control_collection not visible yet, but flagging it so a watch is registered when it becomes visible
   [junit4]   2> 1251051 INFO  (coreLoadExecutor-7105-thread-1-processing-n:127.0.0.1:33749_) [n:127.0.0.1:33749_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader addZkWatch [control_collection]
   [junit4]   2> 1251052 INFO  (coreLoadExecutor-7105-thread-1-processing-n:127.0.0.1:33749_) [n:127.0.0.1:33749_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Deleting data for [control_collection]
   [junit4]   2> 1251052 INFO  (coreLoadExecutor-7105-thread-1-processing-n:127.0.0.1:33749_) [n:127.0.0.1:33749_ c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 1251053 INFO  (OverseerStateUpdate-95844247090429958-127.0.0.1:33749_-n_0000000000) [n:127.0.0.1:33749_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:33749",
   [junit4]   2>   "node_name":"127.0.0.1:33749_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 1251053 INFO  (OverseerStateUpdate-95844247090429958-127.0.0.1:33749_-n_0000000000) [n:127.0.0.1:33749_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:33749",
   [junit4]   2>   "node_name":"127.0.0.1:33749_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 1251053 INFO  (OverseerStateUpdate-95844247090429958-127.0.0.1:33749_-n_0000000000) [n:127.0.0.1:33749_    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 1251053 INFO  (OverseerStateUpdate-95844247090429958-127.0.0.1:33749_-n_0000000000) [n:127.0.0.1:33749_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 1251054 INFO  (zkCallback-30621-thread-1-processing-n:127.0.0.1:33749_) [n:127.0.0.1:33749_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1251054 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [n:127.0.0.1:33749_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1
   [junit4]   2> 1251054 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [n:127.0.0.1:33749_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1252052 INFO  (coreLoadExecutor-7105-thread-1-processing-n:127.0.0.1:33749_) [n:127.0.0.1:33749_ c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1252052 INFO  (coreLoadExecutor-7105-thread-1-processing-n:127.0.0.1:33749_) [n:127.0.0.1:33749_ c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 1252052 INFO  (coreLoadExecutor-7105-thread-1-processing-n:127.0.0.1:33749_) [n:127.0.0.1:33749_ c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 1252053 INFO  (coreLoadExecutor-7105-thread-1-processing-n:127.0.0.1:33749_) [n:127.0.0.1:33749_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/control_collection]
   [junit4]   2> 1252053 INFO  (coreLoadExecutor-7105-thread-1-processing-n:127.0.0.1:33749_) [n:127.0.0.1:33749_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/control_collection] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 1252053 INFO  (coreLoadExecutor-7105-thread-1-processing-n:127.0.0.1:33749_) [n:127.0.0.1:33749_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_E363401D1F6FC7BD-001/control-001/cores/collection1'
   [junit4]   2> 1252053 INFO  (coreLoadExecutor-7105-thread-1-processing-n:127.0.0.1:33749_) [n:127.0.0.1:33749_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1252053 INFO  (coreLoadExecutor-7105-thread-1-processing-n:127.0.0.1:33749_) [n:127.0.0.1:33749_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1252054 INFO  (coreLoadExecutor-7105-thread-1-processing-n:127.0.0.1:33749_) [n:127.0.0.1:33749_ c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 1252057 INFO  (coreLoadExecutor-7105-thread-1-processing-n:127.0.0.1:33749_) [n:127.0.0.1:33749_ c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 1252060 INFO  (coreLoadExecutor-7105-thread-1-processing-n:127.0.0.1:33749_) [n:127.0.0.1:33749_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.1.0
   [junit4]   2> 1252086 INFO  (coreLoadExecutor-7105-thread-1-processing-n:127.0.0.1:33749_) [n:127.0.0.1:33749_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1252090 INFO  (coreLoadExecutor-7105-thread-1-processing-n:127.0.0.1:33749_) [n:127.0.0.1:33749_ c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1252146 INFO  (coreLoadExecutor-7105-thread-1-processing-n:127.0.0.1:33749_) [n:127.0.0.1:33749_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 1252150 INFO  (coreLoadExecutor-7105-thread-1-processing-n:127.0.0.1:33749_) [n:127.0.0.1:33749_ c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 1252151 INFO  (coreLoadExecutor-7105-thread-1-processing-n:127.0.0.1:33749_) [n:127.0.0.1:33749_ c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 1252155 INFO  (coreLoadExecutor-7105-thread-1-processing-n:127.0.0.1:33749_) [n:127.0.0.1:33749_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1252156 INFO  (coreLoadExecutor-7105-thread-1-processing-n:127.0.0.1:33749_) [n:127.0.0.1:33749_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1252157 INFO  (coreLoadExecutor-7105-thread-1-processing-n:127.0.0.1:33749_) [n:127.0.0.1:33749_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1252157 INFO  (coreLoadExecutor-7105-thread-1-processing-n:127.0.0.1:33749_) [n:127.0.0.1:33749_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1252158 INFO  (coreLoadExecutor-7105-thread-1-processing-n:127.0.0.1:33749_) [n:127.0.0.1:33749_ c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1
   [junit4]   2> 1252158 INFO  (coreLoadExecutor-7105-thread-1-processing-n:127.0.0.1:33749_) [n:127.0.0.1:33749_ c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 1252158 INFO  (coreLoadExecutor-7105-thread-1-processing-n:127.0.0.1:33749_) [n:127.0.0.1:33749_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 1252158 INFO  (coreLoadExecutor-7105-thread-1-processing-n:127.0.0.1:33749_) [n:127.0.0.1:33749_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_E363401D1F6FC7BD-001/control-001/cores/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_E363401D1F6FC7BD-001/control-001/cores/collection1/data/]
   [junit4]   2> 1252159 INFO  (coreLoadExecutor-7105-thread-1-processing-n:127.0.0.1:33749_) [n:127.0.0.1:33749_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@6815ed1d
   [junit4]   2> 1252159 INFO  (coreLoadExecutor-7105-thread-1-processing-n:127.0.0.1:33749_) [n:127.0.0.1:33749_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_E363401D1F6FC7BD-001/control-001/cores/collection1/data
   [junit4]   2> 1252159 INFO  (coreLoadExecutor-7105-thread-1-processing-n:127.0.0.1:33749_) [n:127.0.0.1:33749_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_E363401D1F6FC7BD-001/control-001/cores/collection1/data/index/
   [junit4]   2> 1252159 WARN  (coreLoadExecutor-7105-thread-1-processing-n:127.0.0.1:33749_) [n:127.0.0.1:33749_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_E363401D1F6FC7BD-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 1252159 INFO  (coreLoadExecutor-7105-thread-1-processing-n:127.0.0.1:33749_) [n:127.0.0.1:33749_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_E363401D1F6FC7BD-001/control-001/cores/collection1/data/index
   [junit4]   2> 1252160 INFO  (coreLoadExecutor-7105-thread-1-processing-n:127.0.0.1:33749_) [n:127.0.0.1:33749_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=41, maxMergeAtOnceExplicit=41, maxMergedSegmentMB=23.4150390625, floorSegmentMB=1.994140625, forceMergeDeletesPctAllowed=4.78710904714002, segmentsPerTier=44.0, maxCFSSegmentSizeMB=1.3779296875, noCFSRatio=0.361575544003812
   [junit4]   2> 1252183 INFO  (coreLoadExecutor-7105-thread-1-processing-n:127.0.0.1:33749_) [n:127.0.0.1:33749_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(SimpleFSDirectory@/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_E363401D1F6FC7BD-001/index-SimpleFSDirectory-001 lockFactory=org.apache.lucene.store.NativeFSLockFactory@56497789),segFN=segments_1,generation=1}
   [junit4]   2> 1252183 INFO  (coreLoadExecutor-7105-thread-1-processing-n:127.0.0.1:33749_) [n:127.0.0.1:33749_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 1252193 INFO  (coreLoadExecutor-7105-thread-1-processing-n:127.0.0.1:33749_) [n:127.0.0.1:33749_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 1252193 INFO  (coreLoadExecutor-7105-thread-1-processing-n:127.0.0.1:33749_) [n:127.0.0.1:33749_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 1252194 INFO  (coreLoadExecutor-7105-thread-1-processing-n:127.0.0.1:33749_) [n:127.0.0.1:33749_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 1252194 INFO  (coreLoadExecutor-7105-thread-1-processing-n:127.0.0.1:33749_) [n:127.0.0.1:33749_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1252194 INFO  (coreLoadExecutor-7105-thread-1-processing-n:127.0.0.1:33749_) [n:127.0.0.1:33749_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1252194 INFO  (coreLoadExecutor-7105-thread-1-processing-n:127.0.0.1:33749_) [n:127.0.0.1:33749_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 1252194 INFO  (coreLoadExecutor-7105-thread-1-processing-n:127.0.0.1:33749_) [n:127.0.0.1:33749_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1252194 INFO  (coreLoadExecutor-7105-thread-1-processing-n:127.0.0.1:33749_) [n:127.0.0.1:33749_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1252194 INFO  (coreLoadExecutor-7105-thread-1-processing-n:127.0.0.1:33749_) [n:127.0.0.1:33749_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 1252194 WARN  (coreLoadExecutor-7105-thread-1-processing-n:127.0.0.1:33749_) [n:127.0.0.1:33749_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
   [junit4]   2> 1252195 INFO  (coreLoadExecutor-7105-thread-1-processing-n:127.0.0.1:33749_) [n:127.0.0.1:33749_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1252196 INFO  (coreLoadExecutor-7105-thread-1-processing-n:127.0.0.1:33749_) [n:127.0.0.1:33749_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1252196 INFO  (coreLoadExecutor-7105-thread-1-processing-n:127.0.0.1:33749_) [n:127.0.0.1:33749_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1252197 INFO  (coreLoadExecutor-7105-thread-1-processing-n:127.0.0.1:33749_) [n:127.0.0.1:33749_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1252198 INFO  (coreLoadExecutor-7105-thread-1-processing-n:127.0.0.1:33749_) [n:127.0.0.1:33749_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/mbeans,standard,/dump,/update/csv,/sql,/update/json/docs,/admin/luke,/export,/admin/segments,/get,/admin/system,/replication,/admin/properties,/config,/stream,/schema,/admin/plugins,/admin/logging,/update/json,/admin/ping,/admin/threads,/update,/admin/file
   [junit4]   2> 1252199 INFO  (coreLoadExecutor-7105-thread-1-processing-n:127.0.0.1:33749_) [n:127.0.0.1:33749_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 1252202 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:33749_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_E363401D1F6FC7BD-001/control-001/cores/collection1/data/
   [junit4]   2> 1252202 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:33749_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_E363401D1F6FC7BD-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 1252212 INFO  (coreLoadExecutor-7105-thread-1-processing-n:127.0.0.1:33749_) [n:127.0.0.1:33749_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1252212 INFO  (coreLoadExecutor-7105-thread-1-processing-n:127.0.0.1:33749_) [n:127.0.0.1:33749_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1252213 INFO  (coreLoadExecutor-7105-thread-1-processing-n:127.0.0.1:33749_) [n:127.0.0.1:33749_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1252213 INFO  (coreLoadExecutor-7105-thread-1-processing-n:127.0.0.1:33749_) [n:127.0.0.1:33749_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1252213 INFO  (coreLoadExecutor-7105-thread-1-processing-n:127.0.0.1:33749_) [n:127.0.0.1:33749_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@20f9c9aa
   [junit4]   2> 1252214 INFO  (coreLoadExecutor-7105-thread-1-processing-n:127.0.0.1:33749_) [n:127.0.0.1:33749_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(SimpleFSDirectory@/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_E363401D1F6FC7BD-001/index-SimpleFSDirectory-001 lockFactory=org.apache.lucene.store.NativeFSLockFactory@56497789),segFN=segments_1,generation=1}
   [junit4]   2> 1252214 INFO  (coreLoadExecutor-7105-thread-1-processing-n:127.0.0.1:33749_) [n:127.0.0.1:33749_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 1252215 INFO  (coreLoadExecutor-7105-thread-1-processing-n:127.0.0.1:33749_) [n:127.0.0.1:33749_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@3d00526b[collection1] main]
   [junit4]   2> 1252215 INFO  (coreLoadExecutor-7105-thread-1-processing-n:127.0.0.1:33749_) [n:127.0.0.1:33749_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/control_collection]
   [junit4]   2> 1252215 INFO  (coreLoadExecutor-7105-thread-1-processing-n:127.0.0.1:33749_) [n:127.0.0.1:33749_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/control_collection] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 1252215 INFO  (coreLoadExecutor-7105-thread-1-processing-n:127.0.0.1:33749_) [n:127.0.0.1:33749_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 1252216 INFO  (coreLoadExecutor-7105-thread-1-processing-n:127.0.0.1:33749_) [n:127.0.0.1:33749_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1252216 INFO  (coreLoadExecutor-7105-thread-1-processing-n:127.0.0.1:33749_) [n:127.0.0.1:33749_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 1252216 INFO  (coreLoadExecutor-7105-thread-1-processing-n:127.0.0.1:33749_) [n:127.0.0.1:33749_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1252216 INFO  (coreLoadExecutor-7105-thread-1-processing-n:127.0.0.1:33749_) [n:127.0.0.1:33749_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 1252216 INFO  (coreLoadExecutor-7105-thread-1-processing-n:127.0.0.1:33749_) [n:127.0.0.1:33749_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1252216 INFO  (coreLoadExecutor-7105-thread-1-processing-n:127.0.0.1:33749_) [n:127.0.0.1:33749_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 1252216 INFO  (coreLoadExecutor-7105-thread-1-processing-n:127.0.0.1:33749_) [n:127.0.0.1:33749_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1252217 INFO  (searcherExecutor-7106-thread-1-processing-n:127.0.0.1:33749_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:33749_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@3d00526b[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1252217 INFO  (coreLoadExecutor-7105-thread-1-processing-n:127.0.0.1:33749_) [n:127.0.0.1:33749_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 1252217 INFO  (coreLoadExecutor-7105-thread-1-processing-n:127.0.0.1:33749_) [n:127.0.0.1:33749_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 1252217 INFO  (coreLoadExecutor-7105-thread-1-processing-n:127.0.0.1:33749_) [n:127.0.0.1:33749_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 1252217 INFO  (coreLoadExecutor-7105-thread-1-processing-n:127.0.0.1:33749_) [n:127.0.0.1:33749_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1533507955049103360
   [junit4]   2> 1252218 INFO  (coreLoadExecutor-7105-thread-1-processing-n:127.0.0.1:33749_) [n:127.0.0.1:33749_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 1.0ms to seed version buckets with highest version 1533507955049103360
   [junit4]   2> 1252218 INFO  (coreLoadExecutor-7105-thread-1-processing-n:127.0.0.1:33749_) [n:127.0.0.1:33749_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 1252219 INFO  (coreLoadExecutor-7105-thread-1-processing-n:127.0.0.1:33749_) [n:127.0.0.1:33749_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 1252224 INFO  (coreZkRegister-7099-thread-1-processing-n:127.0.0.1:33749_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:33749_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:33749 collection:control_collection shard:shard1
   [junit4]   2> 1252224 INFO  (coreZkRegister-7099-thread-1-processing-n:127.0.0.1:33749_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:33749_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 1252235 INFO  (coreZkRegister-7099-thread-1-processing-n:127.0.0.1:33749_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:33749_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/95844247090429958-core_node1-n_0000000000
   [junit4]   2> 1252236 INFO  (coreZkRegister-7099-thread-1-processing-n:127.0.0.1:33749_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:33749_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard=shard1 and weAreReplacement=false and leaderVoteWait=180000
   [junit4]   2> 1252236 INFO  (OverseerStateUpdate-95844247090429958-127.0.0.1:33749_-n_0000000000) [n:127.0.0.1:33749_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 1252237 INFO  (coreZkRegister-7099-thread-1-processing-n:127.0.0.1:33749_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:33749_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1252237 INFO  (coreZkRegister-7099-thread-1-processing-n:127.0.0.1:33749_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:33749_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1252237 INFO  (coreZkRegister-7099-thread-1-processing-n:127.0.0.1:33749_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:33749_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:33749/collection1/
   [junit4]   2> 1252237 INFO  (coreZkRegister-7099-thread-1-processing-n:127.0.0.1:33749_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:33749_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 1252237 INFO  (coreZkRegister-7099-thread-1-processing-n:127.0.0.1:33749_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:33749_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:33749/collection1/ has no replicas
   [junit4]   2> 1252239 INFO  (coreZkRegister-7099-thread-1-processing-n:127.0.0.1:33749_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:33749_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 1252239 INFO  (coreZkRegister-7099-thread-1-processing-n:127.0.0.1:33749_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:33749_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/control_collection/leaders/shard1/leader after winning as /collections/control_collection/leader_elect/shard1/election/95844247090429958-core_node1-n_0000000000
   [junit4]   2> 1252240 INFO  (coreZkRegister-7099-thread-1-processing-n:127.0.0.1:33749_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:33749_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:33749/collection1/ shard1
   [junit4]   2> 1252240 INFO  (OverseerStateUpdate-95844247090429958-127.0.0.1:33749_-n_0000000000) [n:127.0.0.1:33749_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:33749",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 1
   [junit4]   2> 1252459 INFO  (zkCallback-30621-thread-1-processing-n:127.0.0.1:33749_) [n:127.0.0.1:33749_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1252508 INFO  (coreZkRegister-7099-thread-1-processing-n:127.0.0.1:33749_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:33749_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:33749/collection1/ and leader is http://127.0.0.1:33749/collection1/
   [junit4]   2> 1252508 INFO  (coreZkRegister-7099-thread-1-processing-n:127.0.0.1:33749_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:33749_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:33749
   [junit4]   2> 1252508 INFO  (coreZkRegister-7099-thread-1-processing-n:127.0.0.1:33749_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:33749_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1252508 INFO  (coreZkRegister-7099-thread-1-processing-n:127.0.0.1:33749_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:33749_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 1252508 INFO  (coreZkRegister-7099-thread-1-processing-n:127.0.0.1:33749_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:33749_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1252509 INFO  (OverseerStateUpdate-95844247090429958-127.0.0.1:33749_-n_0000000000) [n:127.0.0.1:33749_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:33749",
   [junit4]   2>   "node_name":"127.0.0.1:33749_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"} current state version: 2
   [junit4]   2> 1252514 INFO  (OverseerStateUpdate-95844247090429958-127.0.0.1:33749_-n_0000000000) [n:127.0.0.1:33749_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:33749",
   [junit4]   2>   "node_name":"127.0.0.1:33749_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 1252555 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.c.s.i.CloudSolrClient Final constructed zkHost string: 127.0.0.1:54018/solr
   [junit4]   2> 1252556 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1252582 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1252609 INFO  (zkCallback-30623-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6b532077 name:ZooKeeperConnection Watcher:127.0.0.1:54018/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1252609 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1252609 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1252609 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1252609 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1252610 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 1252610 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with stateFormat=2
   [junit4]   2> 1252610 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1252615 INFO  (zkCallback-30621-thread-1-processing-n:127.0.0.1:33749_) [n:127.0.0.1:33749_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1252630 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1252630 INFO  (zkCallback-30623-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1252643 INFO  (zkCallback-30624-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@39379216 name:ZooKeeperConnection Watcher:127.0.0.1:54018/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1252644 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1252644 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1252645 INFO  (OverseerStateUpdate-95844247090429958-127.0.0.1:33749_-n_0000000000) [n:127.0.0.1:33749_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"create",
   [junit4]   2>   "name":"collection1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "stateFormat":"2"} current state version: 3
   [junit4]   2> 1252645 INFO  (OverseerStateUpdate-95844247090429958-127.0.0.1:33749_-n_0000000000) [n:127.0.0.1:33749_    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 1252747 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_E363401D1F6FC7BD-001/shard-1-001/cores/collection1
   [junit4]   2> 1252747 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_E363401D1F6FC7BD-001/shard-1-001
   [junit4]   2> 1252748 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.e.j.s.Server jetty-9.3.8.v20160314
   [junit4]   2> 1252752 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@7bc53241{/,null,AVAILABLE}
   [junit4]   2> 1252752 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.e.j.s.ServerConnector Started ServerConnector@231ad273{HTTP/1.1,[http/1.1]}{127.0.0.1:44656}
   [junit4]   2> 1252753 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.e.j.s.Server Started @1254884ms
   [junit4]   2> 1252753 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_E363401D1F6FC7BD-001/tempDir-001/jetty1, solrconfig=solrconfig.xml, hostContext=/, hostPort=44656, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_E363401D1F6FC7BD-001/shard-1-001/cores}
   [junit4]   2> 1252753 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@73d16e93
   [junit4]   2> 1252753 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_E363401D1F6FC7BD-001/shard-1-001'
   [junit4]   2> 1252753 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1252753 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1252753 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1252754 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1252754 INFO  (zkCallback-30625-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3c8b7886 name:ZooKeeperConnection Watcher:127.0.0.1:54018/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1252754 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1252755 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1252755 INFO  (OverseerStateUpdate-95844247090429958-127.0.0.1:33749_-n_0000000000) [n:127.0.0.1:33749_    ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/collection1/state.json
   [junit4]   2> 1252755 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1252755 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_E363401D1F6FC7BD-001/shard-1-001/solr.xml
   [junit4]   2> 1252767 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_E363401D1F6FC7BD-001/shard-1-001/cores
   [junit4]   2> 1252767 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E363401D1F6FC7BD]) [    ] o.a.s.c.CoreContainer New CoreContainer 1387781136
   [junit4]   2> 1252767 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[E3634

[...truncated too long message...]

-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-collective-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-event-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-exception-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-journal-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-normalization-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-operational-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-referral-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-schema-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-subtree-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-trigger-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-jdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-jdbm1-2.0.0-M2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-kerberos-codec-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-ldif-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-mavibot-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-protocol-kerberos-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-protocol-ldap-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-protocol-shared-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-xdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/api-all-1.0.0-M20.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/bcprov-jdk15-1.45.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/cglib-nodep-2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/commons-collections-3.2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/commons-math3-3.4.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/easymock-3.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/ehcache-core-2.4.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/hadoop-common-2.6.0-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/hadoop-hdfs-2.6.0-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/hadoop-minikdc-2.6.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jackson-annotations-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jackson-databind-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jersey-core-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jersey-server-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jetty-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jetty-util-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/mina-core-2.0.0-M5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/mockito-core-1.9.5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/objenesis-1.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/presto-parser-0.122.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/slice-0.10.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/analysis/icu/lucene-analyzers-icu-6.1.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/contrib/solr-analysis-extras/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/queryparser/classes/test:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/contrib/analysis-extras/lib/icu4j-56.1.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-launcher.jar:/x1/jenkins/.ant/lib/ivy-2.3.0.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-antlr.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-regexp.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-resolver.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-net.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jmf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-oro.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-xalan2.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-logging.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jsch.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit4.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-javamail.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jai.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jdepend.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bsf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-swing.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-netrexx.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bcel.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-testutil.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-log4j.jar:/x1/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.8/lib/tools.jar:/x1/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.3.4.jar com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/temp/junit4-J1-20160505_163136_347.events @/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/temp/junit4-J1-20160505_163136_347.suites -stdin
   [junit4] ERROR: JVM J1 ended with an exception: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1501)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:121)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:955)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:952)
   [junit4] 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4] 	at java.lang.Thread.run(Thread.java:745)

BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:747: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:684: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/common-build.xml:529: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/common-build.xml:1457: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/common-build.xml:1014: At least one slave process threw an exception, first: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.

Total time: 245 minutes 24 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
No prior successful build to compare, so performing full copy of artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any




[JENKINS] Lucene-Solr-NightlyTests-6.x - Build # 56 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-6.x/56/

1 tests failed.
FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=17253, name=collection1, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=17253, name=collection1, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:39126: collection already exists: awholynewstresscollection_collection1_0
	at __randomizedtesting.SeedInfo.seed([358C62A1A273FF2D]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:590)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:259)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:248)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:404)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:357)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1192)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:962)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:898)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1595)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1616)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:987)




Build Log:
[...truncated 11942 lines...]
   [junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_358C62A1A273FF2D-001/init-core-data-001
   [junit4]   2> 2210492 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[358C62A1A273FF2D]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true)
   [junit4]   2> 2210492 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[358C62A1A273FF2D]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   2> 2210494 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 2210494 INFO  (Thread-6400) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 2210494 INFO  (Thread-6400) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 2210594 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.ZkTestServer start zk server on port:57646
   [junit4]   2> 2210594 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2210594 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2210596 INFO  (zkCallback-2424-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6d7a83b4 name:ZooKeeperConnection Watcher:127.0.0.1:57646 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2210596 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2210596 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2210596 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 2210597 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2210598 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2210598 INFO  (zkCallback-2425-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@46c47bd5 name:ZooKeeperConnection Watcher:127.0.0.1:57646/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2210598 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2210598 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2210598 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 2210600 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 2210600 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 2210601 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 2210602 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 2210602 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 2210605 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 2210605 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 2210606 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 2210606 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 2210609 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 2210609 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 2210610 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 2210610 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 2210611 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 2210611 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 2210612 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 2210612 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 2210613 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 2210613 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 2210615 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 2210615 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 2210616 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 2210616 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 2210617 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 2210617 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 2210620 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2210621 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2210622 INFO  (zkCallback-2426-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4e49abd5 name:ZooKeeperConnection Watcher:127.0.0.1:57646 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2210622 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2210623 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2210623 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 2210624 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2210624 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2210625 INFO  (zkCallback-2427-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5adfb46a name:ZooKeeperConnection Watcher:127.0.0.1:57646/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2210625 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2210625 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2210625 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf2/solrconfig.xml
   [junit4]   2> 2210625 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/solrconfig.xml
   [junit4]   2> 2210627 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf2/schema.xml
   [junit4]   2> 2210627 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/schema.xml
   [junit4]   2> 2210628 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf2/enumsConfig.xml
   [junit4]   2> 2210628 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/enumsConfig.xml
   [junit4]   2> 2210630 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf2/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 2210630 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 2210631 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf2/stopwords.txt
   [junit4]   2> 2210631 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/stopwords.txt
   [junit4]   2> 2210634 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf2/protwords.txt
   [junit4]   2> 2210634 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/protwords.txt
   [junit4]   2> 2210635 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf2/currency.xml
   [junit4]   2> 2210635 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/currency.xml
   [junit4]   2> 2210638 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf2/open-exchange-rates.json
   [junit4]   2> 2210638 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/open-exchange-rates.json
   [junit4]   2> 2210640 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf2/mapping-ISOLatin1Accent.txt
   [junit4]   2> 2210640 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/mapping-ISOLatin1Accent.txt
   [junit4]   2> 2210641 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf2/old_synonyms.txt
   [junit4]   2> 2210641 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/old_synonyms.txt
   [junit4]   2> 2210642 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf2/synonyms.txt
   [junit4]   2> 2210642 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/synonyms.txt
   [junit4]   2> 2210644 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/elevate.xml to /configs/conf2/elevate.xml
   [junit4]   2> 2210644 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/elevate.xml
   [junit4]   2> 2210704 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_358C62A1A273FF2D-001/control-001/cores/collection1
   [junit4]   2> 2210705 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.e.j.s.Server jetty-9.3.8.v20160314
   [junit4]   2> 2210706 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@2c116764{/,null,AVAILABLE}
   [junit4]   2> 2210708 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.e.j.s.ServerConnector Started ServerConnector@6ecdc687{HTTP/1.1,[http/1.1]}{127.0.0.1:59752}
   [junit4]   2> 2210708 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.e.j.s.Server Started @2212974ms
   [junit4]   2> 2210708 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_358C62A1A273FF2D-001/tempDir-001/control/data, hostContext=/, hostPort=59752, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_358C62A1A273FF2D-001/control-001/cores}
   [junit4]   2> 2210708 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@73d16e93
   [junit4]   2> 2210708 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_358C62A1A273FF2D-001/control-001'
   [junit4]   2> 2210708 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 2210708 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 2210708 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2210709 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2210709 INFO  (zkCallback-2428-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1a550be5 name:ZooKeeperConnection Watcher:127.0.0.1:57646/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2210709 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2210710 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2210710 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 2210710 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_358C62A1A273FF2D-001/control-001/solr.xml
   [junit4]   2> 2210714 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_358C62A1A273FF2D-001/control-001/cores
   [junit4]   2> 2210714 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.CoreContainer New CoreContainer 1685680505
   [junit4]   2> 2210714 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_358C62A1A273FF2D-001/control-001]
   [junit4]   2> 2210714 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_358C62A1A273FF2D-001/control-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_358C62A1A273FF2D-001/control-001/lib
   [junit4]   2> 2210714 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 2210715 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 2210715 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 2210716 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 2210716 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:57646/solr
   [junit4]   2> 2210716 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 2210716 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2210716 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2210719 INFO  (zkCallback-2431-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@10890172 name:ZooKeeperConnection Watcher:127.0.0.1:57646 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2210719 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2210719 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2210720 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [n:127.0.0.1:59752_    ] o.a.s.c.ZkController Added new OnReconnect listener org.apache.solr.cloud.ZkController$$Lambda$27/1828077232@4777a1d9
   [junit4]   2> 2210721 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [n:127.0.0.1:59752_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2210722 INFO  (zkCallback-2432-thread-1-processing-n:127.0.0.1:59752_) [n:127.0.0.1:59752_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5333b36b name:ZooKeeperConnection Watcher:127.0.0.1:57646/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2210722 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [n:127.0.0.1:59752_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2210723 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [n:127.0.0.1:59752_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 2210724 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [n:127.0.0.1:59752_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 2210726 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [n:127.0.0.1:59752_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 2210727 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [n:127.0.0.1:59752_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 2210728 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [n:127.0.0.1:59752_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 2210731 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [n:127.0.0.1:59752_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 2210732 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [n:127.0.0.1:59752_    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 2210733 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [n:127.0.0.1:59752_    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 2210735 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [n:127.0.0.1:59752_    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 2210736 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [n:127.0.0.1:59752_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 2210737 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [n:127.0.0.1:59752_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (0)
   [junit4]   2> 2210739 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [n:127.0.0.1:59752_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 2210740 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [n:127.0.0.1:59752_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 2210741 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [n:127.0.0.1:59752_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 2210741 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [n:127.0.0.1:59752_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/95839401293905926-127.0.0.1:59752_-n_0000000000
   [junit4]   2> 2210741 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [n:127.0.0.1:59752_    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:59752_
   [junit4]   2> 2210741 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [n:127.0.0.1:59752_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 2210742 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [n:127.0.0.1:59752_    ] o.a.s.c.Overseer Overseer (id=95839401293905926-127.0.0.1:59752_-n_0000000000) starting
   [junit4]   2> 2210743 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [n:127.0.0.1:59752_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 2210747 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [n:127.0.0.1:59752_    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 2210747 INFO  (OverseerCollectionConfigSetProcessor-95839401293905926-127.0.0.1:59752_-n_0000000000) [n:127.0.0.1:59752_    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 2210747 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [n:127.0.0.1:59752_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:59752_
   [junit4]   2> 2210748 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [n:127.0.0.1:59752_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:59752_
   [junit4]   2> 2210748 INFO  (OverseerStateUpdate-95839401293905926-127.0.0.1:59752_-n_0000000000) [n:127.0.0.1:59752_    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 2210750 INFO  (zkCallback-2432-thread-1-processing-n:127.0.0.1:59752_) [n:127.0.0.1:59752_    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [0])
   [junit4]   2> 2210757 INFO  (OverseerStateUpdate-95839401293905926-127.0.0.1:59752_-n_0000000000) [n:127.0.0.1:59752_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 2210780 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [n:127.0.0.1:59752_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 2210780 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [n:127.0.0.1:59752_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 2210780 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [n:127.0.0.1:59752_    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_358C62A1A273FF2D-001/control-001/cores
   [junit4]   2> 2210781 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [n:127.0.0.1:59752_    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, loadOnStartup=true, configSetProperties=configsetprops.json, collection=control_collection, coreNodeName=, dataDir=data/, shard=}
   [junit4]   2> 2210781 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [n:127.0.0.1:59752_    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_358C62A1A273FF2D-001/control-001/cores/collection1
   [junit4]   2> 2210781 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [n:127.0.0.1:59752_    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 2210782 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [n:127.0.0.1:59752_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2
   [junit4]   2> 2210782 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [n:127.0.0.1:59752_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 2210784 INFO  (coreLoadExecutor-6016-thread-1-processing-n:127.0.0.1:59752_) [n:127.0.0.1:59752_ c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 2210784 INFO  (coreLoadExecutor-6016-thread-1-processing-n:127.0.0.1:59752_) [n:127.0.0.1:59752_ c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 2210785 INFO  (coreLoadExecutor-6016-thread-1-processing-n:127.0.0.1:59752_) [n:127.0.0.1:59752_ c:control_collection   x:collection1] o.a.s.c.ZkController Collection control_collection not visible yet, but flagging it so a watch is registered when it becomes visible
   [junit4]   2> 2210785 INFO  (coreLoadExecutor-6016-thread-1-processing-n:127.0.0.1:59752_) [n:127.0.0.1:59752_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader addZkWatch [control_collection]
   [junit4]   2> 2210785 INFO  (coreLoadExecutor-6016-thread-1-processing-n:127.0.0.1:59752_) [n:127.0.0.1:59752_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Deleting data for [control_collection]
   [junit4]   2> 2210785 INFO  (coreLoadExecutor-6016-thread-1-processing-n:127.0.0.1:59752_) [n:127.0.0.1:59752_ c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 2210787 INFO  (OverseerStateUpdate-95839401293905926-127.0.0.1:59752_-n_0000000000) [n:127.0.0.1:59752_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:59752",
   [junit4]   2>   "node_name":"127.0.0.1:59752_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 2210787 INFO  (OverseerStateUpdate-95839401293905926-127.0.0.1:59752_-n_0000000000) [n:127.0.0.1:59752_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:59752",
   [junit4]   2>   "node_name":"127.0.0.1:59752_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 2210788 INFO  (OverseerStateUpdate-95839401293905926-127.0.0.1:59752_-n_0000000000) [n:127.0.0.1:59752_    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 2210788 INFO  (OverseerStateUpdate-95839401293905926-127.0.0.1:59752_-n_0000000000) [n:127.0.0.1:59752_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 2210790 INFO  (zkCallback-2432-thread-1-processing-n:127.0.0.1:59752_) [n:127.0.0.1:59752_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 2211787 INFO  (coreLoadExecutor-6016-thread-1-processing-n:127.0.0.1:59752_) [n:127.0.0.1:59752_ c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 2211787 INFO  (coreLoadExecutor-6016-thread-1-processing-n:127.0.0.1:59752_) [n:127.0.0.1:59752_ c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 2211787 INFO  (coreLoadExecutor-6016-thread-1-processing-n:127.0.0.1:59752_) [n:127.0.0.1:59752_ c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 2211787 INFO  (coreLoadExecutor-6016-thread-1-processing-n:127.0.0.1:59752_) [n:127.0.0.1:59752_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/control_collection]
   [junit4]   2> 2211787 INFO  (coreLoadExecutor-6016-thread-1-processing-n:127.0.0.1:59752_) [n:127.0.0.1:59752_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/control_collection] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 2211788 INFO  (coreLoadExecutor-6016-thread-1-processing-n:127.0.0.1:59752_) [n:127.0.0.1:59752_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_358C62A1A273FF2D-001/control-001/cores/collection1'
   [junit4]   2> 2211788 INFO  (coreLoadExecutor-6016-thread-1-processing-n:127.0.0.1:59752_) [n:127.0.0.1:59752_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 2211788 INFO  (coreLoadExecutor-6016-thread-1-processing-n:127.0.0.1:59752_) [n:127.0.0.1:59752_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 2211789 INFO  (coreLoadExecutor-6016-thread-1-processing-n:127.0.0.1:59752_) [n:127.0.0.1:59752_ c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 2211792 INFO  (coreLoadExecutor-6016-thread-1-processing-n:127.0.0.1:59752_) [n:127.0.0.1:59752_ c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 2211796 WARN  (coreLoadExecutor-6016-thread-1-processing-n:127.0.0.1:59752_) [n:127.0.0.1:59752_ c:control_collection   x:collection1] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
   [junit4]   2> 2211797 INFO  (coreLoadExecutor-6016-thread-1-processing-n:127.0.0.1:59752_) [n:127.0.0.1:59752_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.1.0
   [junit4]   2> 2211812 INFO  (coreLoadExecutor-6016-thread-1-processing-n:127.0.0.1:59752_) [n:127.0.0.1:59752_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 2211817 INFO  (coreLoadExecutor-6016-thread-1-processing-n:127.0.0.1:59752_) [n:127.0.0.1:59752_ c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 2211879 INFO  (coreLoadExecutor-6016-thread-1-processing-n:127.0.0.1:59752_) [n:127.0.0.1:59752_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 2211882 INFO  (coreLoadExecutor-6016-thread-1-processing-n:127.0.0.1:59752_) [n:127.0.0.1:59752_ c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 2211890 INFO  (coreLoadExecutor-6016-thread-1-processing-n:127.0.0.1:59752_) [n:127.0.0.1:59752_ c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 2211898 INFO  (coreLoadExecutor-6016-thread-1-processing-n:127.0.0.1:59752_) [n:127.0.0.1:59752_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 2211900 INFO  (coreLoadExecutor-6016-thread-1-processing-n:127.0.0.1:59752_) [n:127.0.0.1:59752_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 2211902 INFO  (coreLoadExecutor-6016-thread-1-processing-n:127.0.0.1:59752_) [n:127.0.0.1:59752_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 2211903 INFO  (coreLoadExecutor-6016-thread-1-processing-n:127.0.0.1:59752_) [n:127.0.0.1:59752_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 2211904 INFO  (coreLoadExecutor-6016-thread-1-processing-n:127.0.0.1:59752_) [n:127.0.0.1:59752_ c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2
   [junit4]   2> 2211904 INFO  (coreLoadExecutor-6016-thread-1-processing-n:127.0.0.1:59752_) [n:127.0.0.1:59752_ c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 2211904 INFO  (coreLoadExecutor-6016-thread-1-processing-n:127.0.0.1:59752_) [n:127.0.0.1:59752_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 2211905 INFO  (coreLoadExecutor-6016-thread-1-processing-n:127.0.0.1:59752_) [n:127.0.0.1:59752_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_358C62A1A273FF2D-001/control-001/cores/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_358C62A1A273FF2D-001/control-001/cores/collection1/data/]
   [junit4]   2> 2211905 INFO  (coreLoadExecutor-6016-thread-1-processing-n:127.0.0.1:59752_) [n:127.0.0.1:59752_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@4ad80101
   [junit4]   2> 2211905 INFO  (coreLoadExecutor-6016-thread-1-processing-n:127.0.0.1:59752_) [n:127.0.0.1:59752_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_358C62A1A273FF2D-001/control-001/cores/collection1/data
   [junit4]   2> 2211905 INFO  (coreLoadExecutor-6016-thread-1-processing-n:127.0.0.1:59752_) [n:127.0.0.1:59752_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_358C62A1A273FF2D-001/control-001/cores/collection1/data/index/
   [junit4]   2> 2211905 WARN  (coreLoadExecutor-6016-thread-1-processing-n:127.0.0.1:59752_) [n:127.0.0.1:59752_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_358C62A1A273FF2D-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 2211906 INFO  (coreLoadExecutor-6016-thread-1-processing-n:127.0.0.1:59752_) [n:127.0.0.1:59752_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_358C62A1A273FF2D-001/control-001/cores/collection1/data/index
   [junit4]   2> 2211906 INFO  (coreLoadExecutor-6016-thread-1-processing-n:127.0.0.1:59752_) [n:127.0.0.1:59752_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=36, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.4045144742146216]
   [junit4]   2> 2211906 INFO  (coreLoadExecutor-6016-thread-1-processing-n:127.0.0.1:59752_) [n:127.0.0.1:59752_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@934c492 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@685c588),segFN=segments_1,generation=1}
   [junit4]   2> 2211906 INFO  (coreLoadExecutor-6016-thread-1-processing-n:127.0.0.1:59752_) [n:127.0.0.1:59752_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 2211907 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:59752_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_358C62A1A273FF2D-001/control-001/cores/collection1/data/
   [junit4]   2> 2211907 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:59752_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_358C62A1A273FF2D-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 2211910 INFO  (coreLoadExecutor-6016-thread-1-processing-n:127.0.0.1:59752_) [n:127.0.0.1:59752_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 2211910 INFO  (coreLoadExecutor-6016-thread-1-processing-n:127.0.0.1:59752_) [n:127.0.0.1:59752_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 2211910 INFO  (coreLoadExecutor-6016-thread-1-processing-n:127.0.0.1:59752_) [n:127.0.0.1:59752_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 2211910 INFO  (coreLoadExecutor-6016-thread-1-processing-n:127.0.0.1:59752_) [n:127.0.0.1:59752_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 2211910 INFO  (coreLoadExecutor-6016-thread-1-processing-n:127.0.0.1:59752_) [n:127.0.0.1:59752_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 2211910 INFO  (coreLoadExecutor-6016-thread-1-processing-n:127.0.0.1:59752_) [n:127.0.0.1:59752_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 2211910 INFO  (coreLoadExecutor-6016-thread-1-processing-n:127.0.0.1:59752_) [n:127.0.0.1:59752_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 2211910 INFO  (coreLoadExecutor-6016-thread-1-processing-n:127.0.0.1:59752_) [n:127.0.0.1:59752_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 2211910 INFO  (coreLoadExecutor-6016-thread-1-processing-n:127.0.0.1:59752_) [n:127.0.0.1:59752_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 2211910 WARN  (coreLoadExecutor-6016-thread-1-processing-n:127.0.0.1:59752_) [n:127.0.0.1:59752_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
   [junit4]   2> 2211912 INFO  (coreLoadExecutor-6016-thread-1-processing-n:127.0.0.1:59752_) [n:127.0.0.1:59752_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2211912 INFO  (coreLoadExecutor-6016-thread-1-processing-n:127.0.0.1:59752_) [n:127.0.0.1:59752_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2211913 INFO  (coreLoadExecutor-6016-thread-1-processing-n:127.0.0.1:59752_) [n:127.0.0.1:59752_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2211914 INFO  (coreLoadExecutor-6016-thread-1-processing-n:127.0.0.1:59752_) [n:127.0.0.1:59752_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2211916 INFO  (coreLoadExecutor-6016-thread-1-processing-n:127.0.0.1:59752_) [n:127.0.0.1:59752_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/mbeans,standard,/dump,/update/csv,/sql,/update/json/docs,/admin/luke,/export,/admin/segments,/get,/admin/system,/replication,/admin/properties,/config,/stream,/schema,/admin/plugins,/admin/logging,/update/json,/admin/ping,/admin/threads,/update,/admin/file
   [junit4]   2> 2211916 INFO  (coreLoadExecutor-6016-thread-1-processing-n:127.0.0.1:59752_) [n:127.0.0.1:59752_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 2211917 INFO  (coreLoadExecutor-6016-thread-1-processing-n:127.0.0.1:59752_) [n:127.0.0.1:59752_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 2211917 INFO  (coreLoadExecutor-6016-thread-1-processing-n:127.0.0.1:59752_) [n:127.0.0.1:59752_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 2211918 INFO  (coreLoadExecutor-6016-thread-1-processing-n:127.0.0.1:59752_) [n:127.0.0.1:59752_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 2211918 INFO  (coreLoadExecutor-6016-thread-1-processing-n:127.0.0.1:59752_) [n:127.0.0.1:59752_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 2211918 INFO  (coreLoadExecutor-6016-thread-1-processing-n:127.0.0.1:59752_) [n:127.0.0.1:59752_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=42, maxMergeAtOnceExplicit=21, maxMergedSegmentMB=61.1064453125, floorSegmentMB=0.953125, forceMergeDeletesPctAllowed=29.561958091259484, segmentsPerTier=49.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
   [junit4]   2> 2211918 INFO  (coreLoadExecutor-6016-thread-1-processing-n:127.0.0.1:59752_) [n:127.0.0.1:59752_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@934c492 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@685c588),segFN=segments_1,generation=1}
   [junit4]   2> 2211918 INFO  (coreLoadExecutor-6016-thread-1-processing-n:127.0.0.1:59752_) [n:127.0.0.1:59752_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 2211918 INFO  (coreLoadExecutor-6016-thread-1-processing-n:127.0.0.1:59752_) [n:127.0.0.1:59752_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@6bbff668[collection1] main]
   [junit4]   2> 2211919 INFO  (coreLoadExecutor-6016-thread-1-processing-n:127.0.0.1:59752_) [n:127.0.0.1:59752_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/control_collection]
   [junit4]   2> 2211919 INFO  (coreLoadExecutor-6016-thread-1-processing-n:127.0.0.1:59752_) [n:127.0.0.1:59752_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/control_collection] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 2211919 INFO  (coreLoadExecutor-6016-thread-1-processing-n:127.0.0.1:59752_) [n:127.0.0.1:59752_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 2211920 INFO  (coreLoadExecutor-6016-thread-1-processing-n:127.0.0.1:59752_) [n:127.0.0.1:59752_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 2211920 INFO  (coreLoadExecutor-6016-thread-1-processing-n:127.0.0.1:59752_) [n:127.0.0.1:59752_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 2211920 INFO  (coreLoadExecutor-6016-thread-1-processing-n:127.0.0.1:59752_) [n:127.0.0.1:59752_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 2211920 INFO  (coreLoadExecutor-6016-thread-1-processing-n:127.0.0.1:59752_) [n:127.0.0.1:59752_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 2211920 INFO  (coreLoadExecutor-6016-thread-1-processing-n:127.0.0.1:59752_) [n:127.0.0.1:59752_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 2211920 INFO  (coreLoadExecutor-6016-thread-1-processing-n:127.0.0.1:59752_) [n:127.0.0.1:59752_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 2211920 INFO  (coreLoadExecutor-6016-thread-1-processing-n:127.0.0.1:59752_) [n:127.0.0.1:59752_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 2211921 INFO  (searcherExecutor-6017-thread-1-processing-n:127.0.0.1:59752_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:59752_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@6bbff668[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2211921 INFO  (coreLoadExecutor-6016-thread-1-processing-n:127.0.0.1:59752_) [n:127.0.0.1:59752_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 2211921 INFO  (coreLoadExecutor-6016-thread-1-processing-n:127.0.0.1:59752_) [n:127.0.0.1:59752_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 2211921 INFO  (coreLoadExecutor-6016-thread-1-processing-n:127.0.0.1:59752_) [n:127.0.0.1:59752_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 2211921 INFO  (coreLoadExecutor-6016-thread-1-processing-n:127.0.0.1:59752_) [n:127.0.0.1:59752_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1533430422197764096
   [junit4]   2> 2211924 INFO  (coreLoadExecutor-6016-thread-1-processing-n:127.0.0.1:59752_) [n:127.0.0.1:59752_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 3.0ms to seed version buckets with highest version 1533430422197764096
   [junit4]   2> 2211924 INFO  (coreLoadExecutor-6016-thread-1-processing-n:127.0.0.1:59752_) [n:127.0.0.1:59752_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 2211925 INFO  (coreLoadExecutor-6016-thread-1-processing-n:127.0.0.1:59752_) [n:127.0.0.1:59752_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 2211925 INFO  (coreZkRegister-6010-thread-1-processing-n:127.0.0.1:59752_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:59752_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:59752 collection:control_collection shard:shard1
   [junit4]   2> 2211926 INFO  (coreZkRegister-6010-thread-1-processing-n:127.0.0.1:59752_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:59752_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 2211928 INFO  (coreZkRegister-6010-thread-1-processing-n:127.0.0.1:59752_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:59752_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/95839401293905926-core_node1-n_0000000000
   [junit4]   2> 2211928 INFO  (coreZkRegister-6010-thread-1-processing-n:127.0.0.1:59752_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:59752_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard=shard1 and weAreReplacement=false and leaderVoteWait=180000
   [junit4]   2> 2211929 INFO  (coreZkRegister-6010-thread-1-processing-n:127.0.0.1:59752_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:59752_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 2211929 INFO  (coreZkRegister-6010-thread-1-processing-n:127.0.0.1:59752_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:59752_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 2211929 INFO  (coreZkRegister-6010-thread-1-processing-n:127.0.0.1:59752_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:59752_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:59752/collection1/
   [junit4]   2> 2211929 INFO  (coreZkRegister-6010-thread-1-processing-n:127.0.0.1:59752_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:59752_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 2211929 INFO  (coreZkRegister-6010-thread-1-processing-n:127.0.0.1:59752_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:59752_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:59752/collection1/ has no replicas
   [junit4]   2> 2211929 INFO  (OverseerStateUpdate-95839401293905926-127.0.0.1:59752_-n_0000000000) [n:127.0.0.1:59752_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 2211930 INFO  (coreZkRegister-6010-thread-1-processing-n:127.0.0.1:59752_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:59752_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 2211931 INFO  (coreZkRegister-6010-thread-1-processing-n:127.0.0.1:59752_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:59752_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/control_collection/leaders/shard1/leader after winning as /collections/control_collection/leader_elect/shard1/election/95839401293905926-core_node1-n_0000000000
   [junit4]   2> 2211934 INFO  (coreZkRegister-6010-thread-1-processing-n:127.0.0.1:59752_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:59752_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:59752/collection1/ shard1
   [junit4]   2> 2211934 INFO  (OverseerStateUpdate-95839401293905926-127.0.0.1:59752_-n_0000000000) [n:127.0.0.1:59752_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:59752",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 1
   [junit4]   2> 2212036 INFO  (zkCallback-2432-thread-1-processing-n:127.0.0.1:59752_) [n:127.0.0.1:59752_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 2212085 INFO  (coreZkRegister-6010-thread-1-processing-n:127.0.0.1:59752_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:59752_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:59752/collection1/ and leader is http://127.0.0.1:59752/collection1/
   [junit4]   2> 2212085 INFO  (coreZkRegister-6010-thread-1-processing-n:127.0.0.1:59752_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:59752_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:59752
   [junit4]   2> 2212085 INFO  (coreZkRegister-6010-thread-1-processing-n:127.0.0.1:59752_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:59752_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 2212085 INFO  (coreZkRegister-6010-thread-1-processing-n:127.0.0.1:59752_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:59752_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 2212085 INFO  (coreZkRegister-6010-thread-1-processing-n:127.0.0.1:59752_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:59752_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 2212086 INFO  (OverseerStateUpdate-95839401293905926-127.0.0.1:59752_-n_0000000000) [n:127.0.0.1:59752_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:59752",
   [junit4]   2>   "node_name":"127.0.0.1:59752_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"} current state version: 2
   [junit4]   2> 2212087 INFO  (OverseerStateUpdate-95839401293905926-127.0.0.1:59752_-n_0000000000) [n:127.0.0.1:59752_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:59752",
   [junit4]   2>   "node_name":"127.0.0.1:59752_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 2212188 INFO  (zkCallback-2432-thread-1-processing-n:127.0.0.1:59752_) [n:127.0.0.1:59752_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 2212284 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2212284 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2212285 INFO  (zkCallback-2434-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@408451f4 name:ZooKeeperConnection Watcher:127.0.0.1:57646/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2212285 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2212285 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2212285 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 2212290 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 2212290 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 2212346 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_358C62A1A273FF2D-001/shard-1-001/cores/collection1
   [junit4]   2> 2212347 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_358C62A1A273FF2D-001/shard-1-001
   [junit4]   2> 2212347 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.e.j.s.Server jetty-9.3.8.v20160314
   [junit4]   2> 2212348 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@ac7d460{/,null,AVAILABLE}
   [junit4]   2> 2212349 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.e.j.s.ServerConnector Started ServerConnector@18a78c88{HTTP/1.1,[http/1.1]}{127.0.0.1:46267}
   [junit4]   2> 2212349 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.e.j.s.Server Started @2214615ms
   [junit4]   2> 2212349 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_358C62A1A273FF2D-001/tempDir-001/jetty1, solrconfig=solrconfig.xml, hostContext=/, hostPort=46267, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_358C62A1A273FF2D-001/shard-1-001/cores}
   [junit4]   2> 2212349 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@73d16e93
   [junit4]   2> 2212349 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_358C62A1A273FF2D-001/shard-1-001'
   [junit4]   2> 2212350 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 2212350 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 2212350 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2212350 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2212351 INFO  (zkCallback-2435-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@974b456 name:ZooKeeperConnection Watcher:127.0.0.1:57646/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2212351 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2212351 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2212355 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 2212355 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_358C62A1A273FF2D-001/shard-1-001/solr.xml
   [junit4]   2> 2212358 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_358C62A1A273FF2D-001/shard-1-001/cores
   [junit4]   2> 2212358 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.CoreContainer New CoreContainer 351112566
   [junit4]   2> 2212358 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_358C62A1A273FF2D-001/shard-1-001]
   [junit4]   2> 2212358 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_358C62A1A273FF2D-001/shard-1-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_358C62A1A273FF2D-001/shard-1-001/lib
   [junit4]   2> 2212359 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 2212359 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 2212360 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 2212360 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 2212360 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:57646/solr
   [junit4]   2> 2212360 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 2212360 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2212360 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2212361 INFO  (zkCallback-2438-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1ce2ddd6 name:ZooKeeperConnection Watcher:127.0.0.1:57646 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2212361 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2212361 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2212362 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [n:127.0.0.1:46267_    ] o.a.s.c.ZkController Added new OnReconnect listener org.apache.solr.cloud.ZkController$$Lambda$27/1828077232@6a2831d8
   [junit4]   2> 2212362 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [n:127.0.0.1:46267_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2212363 INFO  (zkCallback-2439-thread-1-processing-n:127.0.0.1:46267_) [n:127.0.0.1:46267_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6095ac80 name:ZooKeeperConnection Watcher:127.0.0.1:57646/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2212363 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [n:127.0.0.1:46267_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2212367 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [n:127.0.0.1:46267_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 2212368 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [n:127.0.0.1:46267_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 2212370 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [n:127.0.0.1:46267_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 2212370 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [n:127.0.0.1:46267_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/95839401293905930-127.0.0.1:46267_-n_0000000001
   [junit4]   2> 2212374 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [n:127.0.0.1:46267_    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/95839401293905926-127.0.0.1:59752_-n_0000000000 to know if I could be the leader
   [junit4]   2> 2212374 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [n:127.0.0.1:46267_    ] o.a.s.c.ZkController Publish node=127.0.0.1:46267_ as DOWN
   [junit4]   2> 2212376 INFO  (OverseerStateUpdate-95839401293905926-127.0.0.1:59752_-n_0000000000) [n:127.0.0.1:59752_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"downnode",
   [junit4]   2>   "node_name":"127.0.0.1:46267_"} current state version: 3
   [junit4]   2> 2212376 INFO  (OverseerStateUpdate-95839401293905926-127.0.0.1:59752_-n_0000000000) [n:127.0.0.1:59752_    ] o.a.s.c.o.NodeMutator DownNode state invoked for node: 127.0.0.1:46267_
   [junit4]   2> 2212478 INFO  (zkCallback-2434-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 2212479 INFO  (zkCallback-2432-thread-1-processing-n:127.0.0.1:59752_) [n:127.0.0.1:59752_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 2212479 INFO  (zkCallback-2439-thread-1-processing-n:127.0.0.1:46267_) [n:127.0.0.1:46267_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 2213376 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [n:127.0.0.1:46267_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:46267_
   [junit4]   2> 2213377 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [n:127.0.0.1:46267_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:46267_
   [junit4]   2> 2213379 INFO  (zkCallback-2439-thread-1-processing-n:127.0.0.1:46267_) [n:127.0.0.1:46267_    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 2213379 INFO  (zkCallback-2434-thread-1) [    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 2213379 INFO  (zkCallback-2432-thread-2-processing-n:127.0.0.1:59752_) [n:127.0.0.1:59752_    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 2213379 INFO  (zkCallback-2434-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 2213379 INFO  (zkCallback-2439-thread-1-processing-n:127.0.0.1:46267_) [n:127.0.0.1:46267_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 2213381 INFO  (zkCallback-2432-thread-2-processing-n:127.0.0.1:59752_) [n:127.0.0.1:59752_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 2213387 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [n:127.0.0.1:46267_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 2213387 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [n:127.0.0.1:46267_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 2213387 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [n:127.0.0.1:46267_    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_358C62A1A273FF2D-001/shard-1-001/cores
   [junit4]   2> 2213388 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [n:127.0.0.1:46267_    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, loadOnStartup=true, configSetProperties=configsetprops.json, collection=collection1, coreNodeName=, dataDir=data/, shard=}
   [junit4]   2> 2213388 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [n:127.0.0.1:46267_    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_358C62A1A273FF2D-001/shard-1-001/cores/collection1
   [junit4]   2> 2213388 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [n:127.0.0.1:46267_    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 2213389 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [n:127.0.0.1:46267_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2
   [junit4]   2> 2213389 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[358C62A1A273FF2D]) [n:127.0.0.1:46267_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 2213389 INFO  (coreLoadExecutor-6027-thread-1-processing-n:127.0.0.1:46267_) [n:127.0.0.1:46267_ c:collection1   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 2213390 INFO  (coreLoadExecutor-6027-thread-1-processing-n:127.0.0.1:46267_) [n:127.0.0.1:46267_ c:collection1   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 2213390 INFO  (coreLoadExecutor-6027-thread-1-processing-n:127.0.0.1:46267_) [n:127.0.0.1:46267_ c:collection1   x:collection1] o.a.s.c.ZkController Collection collection1 not visible yet, but flagging it so a watch is registered when it becomes visible
   [junit4]   2> 2213390 INFO  (coreLoadExecutor-6027-thread-1-processing-n:127.0.0.1:46267_) [n:127.0.0.1:46267_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader addZkWatch [collection1]
   [junit4]   2> 2213390 INFO  (OverseerStateUpdate-95839401293905926-127.0.0.1:59752_-n_0000000000) [n:127.0.0.1:59752_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:46267",
   [junit4]   2>   "node_name":"127.0.0.1:46267_",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "operation":"state"} current state version: 4
   [junit4]   2> 2213390 INFO  (coreLoadExecutor-6027-thread-1-processing-n:127.0.0.1:46267_) [n:127.0.0.1:46267_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Deleting data for [collection1]
   [junit4]   2> 2213390 INFO  (OverseerStateUpdate-95839401293905926-127.0.0.1:59752_-n_0000000000) [n:127.0.0.1:59752_    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:46267",
   [junit4]   2>   "node_name":"127.0.0.1:46267_",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 2213391 INFO  (OverseerStateUpdate-95839401293905926-127.0.0.1:59752_-n_0000000000) [n:127.0.0.1:59752_    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 2213391 INFO  (OverseerStateUpdate-95839401293905926-127.0.0.1:59752_-n_0000000000) [n:127.0.0.1:59752_    ] o.a.s.c.o.ReplicaMutator Assigni

[...truncated too long message...]

-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-collective-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-event-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-exception-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-journal-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-normalization-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-operational-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-referral-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-schema-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-subtree-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-trigger-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-jdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-jdbm1-2.0.0-M2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-kerberos-codec-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-ldif-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-mavibot-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-protocol-kerberos-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-protocol-ldap-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-protocol-shared-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-xdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/api-all-1.0.0-M20.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/bcprov-jdk15-1.45.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/cglib-nodep-2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/commons-collections-3.2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/commons-math3-3.4.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/easymock-3.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/ehcache-core-2.4.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/hadoop-common-2.6.0-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/hadoop-hdfs-2.6.0-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/hadoop-minikdc-2.6.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jackson-annotations-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jackson-databind-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jersey-core-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jersey-server-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jetty-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jetty-util-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/mina-core-2.0.0-M5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/mockito-core-1.9.5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/objenesis-1.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/presto-parser-0.122.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/slice-0.10.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/analysis/icu/lucene-analyzers-icu-6.1.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/contrib/solr-analysis-extras/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/queryparser/classes/test:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/contrib/analysis-extras/lib/icu4j-56.1.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-launcher.jar:/x1/jenkins/.ant/lib/ivy-2.3.0.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-antlr.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-regexp.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-resolver.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-net.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jmf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-oro.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-xalan2.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-logging.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jsch.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit4.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-javamail.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jai.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jdepend.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bsf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-swing.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-netrexx.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bcel.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-testutil.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-log4j.jar:/x1/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.8/lib/tools.jar:/x1/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.3.4.jar com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/temp/junit4-J0-20160504_194315_416.events @/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/temp/junit4-J0-20160504_194315_416.suites -stdin
   [junit4] ERROR: JVM J0 ended with an exception: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1501)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:121)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:955)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:952)
   [junit4] 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4] 	at java.lang.Thread.run(Thread.java:745)

BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:747: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:684: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/common-build.xml:529: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/common-build.xml:1457: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/common-build.xml:1014: At least one slave process threw an exception, first: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.

Total time: 233 minutes 53 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
No prior successful build to compare, so performing full copy of artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any




[JENKINS] Lucene-Solr-NightlyTests-6.x - Build # 55 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-6.x/55/

2 tests failed.
FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Timeout occured while waiting response from server at: http://127.0.0.1:59307

Stack Trace:
org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:59307
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:601)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:259)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:248)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.makeRequest(CollectionsAPIDistributedZkTest.java:399)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testErrorHandling(CollectionsAPIDistributedZkTest.java:457)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test(CollectionsAPIDistributedZkTest.java:179)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.net.SocketTimeoutException: Read timed out
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
	at java.net.SocketInputStream.read(SocketInputStream.java:170)
	at java.net.SocketInputStream.read(SocketInputStream.java:141)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:160)
	at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:84)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:273)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
	at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
	at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)
	at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)
	at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197)
	at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)
	at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)
	at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685)
	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487)
	at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:495)
	... 47 more


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest

Error Message:
ObjectTracker found 1 object(s) that were not released!!! [HdfsTransactionLog]

Stack Trace:
java.lang.AssertionError: ObjectTracker found 1 object(s) that were not released!!! [HdfsTransactionLog]
	at __randomizedtesting.SeedInfo.seed([C8145CEB159DF6BE]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.junit.Assert.assertNull(Assert.java:551)
	at org.apache.solr.SolrTestCaseJ4.afterClass(SolrTestCaseJ4.java:256)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:834)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 12500 lines...]
   [junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_C8145CEB159DF6BE-001/init-core-data-001
   [junit4]   2> 1706562 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[C8145CEB159DF6BE]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true)
   [junit4]   2> 1706562 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[C8145CEB159DF6BE]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   2> 1706564 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 1706565 INFO  (Thread-22087) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1706565 INFO  (Thread-22087) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 1706665 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.ZkTestServer start zk server on port:56141
   [junit4]   2> 1706665 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1706670 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1706672 INFO  (zkCallback-1204-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@73dcb03 name:ZooKeeperConnection Watcher:127.0.0.1:56141 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1706673 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1706674 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1706674 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 1706677 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1706683 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1706692 INFO  (zkCallback-1205-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7e913cbb name:ZooKeeperConnection Watcher:127.0.0.1:56141/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1706692 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1706693 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1706693 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 1706694 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 1706695 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 1706696 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 1706697 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 1706697 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 1706711 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 1706711 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 1706715 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1706715 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1706718 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 1706718 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 1706722 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 1706722 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 1706731 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 1706731 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 1706734 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 1706734 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 1706736 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1706736 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1706737 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1706738 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1706739 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 1706739 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 1706740 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 1706741 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 1706743 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1706743 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1706744 INFO  (zkCallback-1206-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@527bb8de name:ZooKeeperConnection Watcher:127.0.0.1:56141 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1706746 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1706746 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1706746 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 1706749 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1706755 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1706757 INFO  (zkCallback-1207-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@26b9412d name:ZooKeeperConnection Watcher:127.0.0.1:56141/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1706757 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1706757 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1706758 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf2/solrconfig.xml
   [junit4]   2> 1706758 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/solrconfig.xml
   [junit4]   2> 1706761 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf2/schema.xml
   [junit4]   2> 1706761 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/schema.xml
   [junit4]   2> 1706767 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf2/enumsConfig.xml
   [junit4]   2> 1706767 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/enumsConfig.xml
   [junit4]   2> 1706769 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf2/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1706769 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1706770 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf2/stopwords.txt
   [junit4]   2> 1706770 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/stopwords.txt
   [junit4]   2> 1706772 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf2/protwords.txt
   [junit4]   2> 1706773 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/protwords.txt
   [junit4]   2> 1706775 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf2/currency.xml
   [junit4]   2> 1706775 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/currency.xml
   [junit4]   2> 1706777 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf2/open-exchange-rates.json
   [junit4]   2> 1706777 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/open-exchange-rates.json
   [junit4]   2> 1706780 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf2/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1706780 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1706783 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf2/old_synonyms.txt
   [junit4]   2> 1706783 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/old_synonyms.txt
   [junit4]   2> 1706793 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf2/synonyms.txt
   [junit4]   2> 1706794 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/synonyms.txt
   [junit4]   2> 1706796 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/elevate.xml to /configs/conf2/elevate.xml
   [junit4]   2> 1706796 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/elevate.xml
   [junit4]   2> 1706950 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_C8145CEB159DF6BE-001/control-001/cores/collection1
   [junit4]   2> 1706952 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.e.j.s.Server jetty-9.3.8.v20160314
   [junit4]   2> 1706956 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@e0fa46d{/,null,AVAILABLE}
   [junit4]   2> 1706957 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.e.j.s.ServerConnector Started ServerConnector@4ed541c9{HTTP/1.1,[http/1.1]}{127.0.0.1:35045}
   [junit4]   2> 1706957 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.e.j.s.Server Started @1710103ms
   [junit4]   2> 1706957 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_C8145CEB159DF6BE-001/tempDir-001/control/data, hostContext=/, hostPort=35045, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_C8145CEB159DF6BE-001/control-001/cores}
   [junit4]   2> 1706957 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@73d16e93
   [junit4]   2> 1706957 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_C8145CEB159DF6BE-001/control-001'
   [junit4]   2> 1706957 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1706957 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1706957 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1706964 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1706971 INFO  (zkCallback-1208-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@253e2529 name:ZooKeeperConnection Watcher:127.0.0.1:56141/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1706971 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1706971 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1706972 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1706972 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_C8145CEB159DF6BE-001/control-001/solr.xml
   [junit4]   2> 1706977 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_C8145CEB159DF6BE-001/control-001/cores
   [junit4]   2> 1706977 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.CoreContainer New CoreContainer 170742918
   [junit4]   2> 1706977 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_C8145CEB159DF6BE-001/control-001]
   [junit4]   2> 1706977 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_C8145CEB159DF6BE-001/control-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_C8145CEB159DF6BE-001/control-001/lib
   [junit4]   2> 1706978 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1706979 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1706980 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1706980 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1706980 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:56141/solr
   [junit4]   2> 1706980 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1706980 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1706983 INFO  (zkCallback-1211-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5be943c0 name:ZooKeeperConnection Watcher:127.0.0.1:56141 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1706987 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1706987 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1706987 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1706988 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [n:127.0.0.1:35045_    ] o.a.s.c.ZkController Added new OnReconnect listener org.apache.solr.cloud.ZkController$$Lambda$47/1525438528@2718f158
   [junit4]   2> 1706988 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [n:127.0.0.1:35045_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1706989 INFO  (zkCallback-1212-thread-1-processing-n:127.0.0.1:35045_) [n:127.0.0.1:35045_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@69187537 name:ZooKeeperConnection Watcher:127.0.0.1:56141/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1706989 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [n:127.0.0.1:35045_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1706990 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [n:127.0.0.1:35045_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 1706992 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [n:127.0.0.1:35045_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 1706995 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [n:127.0.0.1:35045_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 1706997 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [n:127.0.0.1:35045_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 1706999 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [n:127.0.0.1:35045_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 1707001 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [n:127.0.0.1:35045_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 1707002 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [n:127.0.0.1:35045_    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 1707004 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [n:127.0.0.1:35045_    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 1707005 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [n:127.0.0.1:35045_    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 1707005 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [n:127.0.0.1:35045_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1707006 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [n:127.0.0.1:35045_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (0)
   [junit4]   2> 1707008 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [n:127.0.0.1:35045_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 1707009 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [n:127.0.0.1:35045_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 1707010 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [n:127.0.0.1:35045_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1707010 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [n:127.0.0.1:35045_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/95834174887100422-127.0.0.1:35045_-n_0000000000
   [junit4]   2> 1707011 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [n:127.0.0.1:35045_    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:35045_
   [junit4]   2> 1707011 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [n:127.0.0.1:35045_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 1707012 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [n:127.0.0.1:35045_    ] o.a.s.c.Overseer Overseer (id=95834174887100422-127.0.0.1:35045_-n_0000000000) starting
   [junit4]   2> 1707013 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [n:127.0.0.1:35045_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 1707018 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [n:127.0.0.1:35045_    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 1707019 INFO  (OverseerCollectionConfigSetProcessor-95834174887100422-127.0.0.1:35045_-n_0000000000) [n:127.0.0.1:35045_    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 1707019 INFO  (OverseerStateUpdate-95834174887100422-127.0.0.1:35045_-n_0000000000) [n:127.0.0.1:35045_    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 1707019 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [n:127.0.0.1:35045_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:35045_
   [junit4]   2> 1707021 INFO  (OverseerStateUpdate-95834174887100422-127.0.0.1:35045_-n_0000000000) [n:127.0.0.1:35045_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (0)
   [junit4]   2> 1707021 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [n:127.0.0.1:35045_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:35045_
   [junit4]   2> 1707022 INFO  (zkCallback-1212-thread-1-processing-n:127.0.0.1:35045_) [n:127.0.0.1:35045_    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [0])
   [junit4]   2> 1707022 INFO  (zkCallback-1212-thread-1-processing-n:127.0.0.1:35045_) [n:127.0.0.1:35045_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1707037 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [n:127.0.0.1:35045_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1707037 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [n:127.0.0.1:35045_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1707038 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [n:127.0.0.1:35045_    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_C8145CEB159DF6BE-001/control-001/cores
   [junit4]   2> 1707039 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [n:127.0.0.1:35045_    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, loadOnStartup=true, configSetProperties=configsetprops.json, collection=control_collection, coreNodeName=, dataDir=data/, shard=}
   [junit4]   2> 1707039 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [n:127.0.0.1:35045_    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_C8145CEB159DF6BE-001/control-001/cores/collection1
   [junit4]   2> 1707039 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [n:127.0.0.1:35045_    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 1707043 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [n:127.0.0.1:35045_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1
   [junit4]   2> 1707043 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [n:127.0.0.1:35045_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1707043 INFO  (coreLoadExecutor-3921-thread-1-processing-n:127.0.0.1:35045_) [n:127.0.0.1:35045_ c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 1707043 INFO  (coreLoadExecutor-3921-thread-1-processing-n:127.0.0.1:35045_) [n:127.0.0.1:35045_ c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1707044 INFO  (coreLoadExecutor-3921-thread-1-processing-n:127.0.0.1:35045_) [n:127.0.0.1:35045_ c:control_collection   x:collection1] o.a.s.c.ZkController Collection control_collection not visible yet, but flagging it so a watch is registered when it becomes visible
   [junit4]   2> 1707044 INFO  (coreLoadExecutor-3921-thread-1-processing-n:127.0.0.1:35045_) [n:127.0.0.1:35045_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader addZkWatch [control_collection]
   [junit4]   2> 1707045 INFO  (OverseerStateUpdate-95834174887100422-127.0.0.1:35045_-n_0000000000) [n:127.0.0.1:35045_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:35045",
   [junit4]   2>   "node_name":"127.0.0.1:35045_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 1707045 INFO  (coreLoadExecutor-3921-thread-1-processing-n:127.0.0.1:35045_) [n:127.0.0.1:35045_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Deleting data for [control_collection]
   [junit4]   2> 1707045 INFO  (OverseerStateUpdate-95834174887100422-127.0.0.1:35045_-n_0000000000) [n:127.0.0.1:35045_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:35045",
   [junit4]   2>   "node_name":"127.0.0.1:35045_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 1707046 INFO  (coreLoadExecutor-3921-thread-1-processing-n:127.0.0.1:35045_) [n:127.0.0.1:35045_ c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 1707046 INFO  (OverseerStateUpdate-95834174887100422-127.0.0.1:35045_-n_0000000000) [n:127.0.0.1:35045_    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 1707046 INFO  (OverseerStateUpdate-95834174887100422-127.0.0.1:35045_-n_0000000000) [n:127.0.0.1:35045_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 1707047 INFO  (zkCallback-1212-thread-1-processing-n:127.0.0.1:35045_) [n:127.0.0.1:35045_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1708046 INFO  (coreLoadExecutor-3921-thread-1-processing-n:127.0.0.1:35045_) [n:127.0.0.1:35045_ c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1708046 INFO  (coreLoadExecutor-3921-thread-1-processing-n:127.0.0.1:35045_) [n:127.0.0.1:35045_ c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 1708047 INFO  (coreLoadExecutor-3921-thread-1-processing-n:127.0.0.1:35045_) [n:127.0.0.1:35045_ c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 1708047 INFO  (coreLoadExecutor-3921-thread-1-processing-n:127.0.0.1:35045_) [n:127.0.0.1:35045_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/control_collection]
   [junit4]   2> 1708048 INFO  (coreLoadExecutor-3921-thread-1-processing-n:127.0.0.1:35045_) [n:127.0.0.1:35045_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/control_collection] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 1708048 INFO  (coreLoadExecutor-3921-thread-1-processing-n:127.0.0.1:35045_) [n:127.0.0.1:35045_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_C8145CEB159DF6BE-001/control-001/cores/collection1'
   [junit4]   2> 1708048 INFO  (coreLoadExecutor-3921-thread-1-processing-n:127.0.0.1:35045_) [n:127.0.0.1:35045_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1708048 INFO  (coreLoadExecutor-3921-thread-1-processing-n:127.0.0.1:35045_) [n:127.0.0.1:35045_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1708050 INFO  (coreLoadExecutor-3921-thread-1-processing-n:127.0.0.1:35045_) [n:127.0.0.1:35045_ c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 1708054 INFO  (coreLoadExecutor-3921-thread-1-processing-n:127.0.0.1:35045_) [n:127.0.0.1:35045_ c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 1708058 INFO  (coreLoadExecutor-3921-thread-1-processing-n:127.0.0.1:35045_) [n:127.0.0.1:35045_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.1.0
   [junit4]   2> 1708066 INFO  (coreLoadExecutor-3921-thread-1-processing-n:127.0.0.1:35045_) [n:127.0.0.1:35045_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1708071 INFO  (coreLoadExecutor-3921-thread-1-processing-n:127.0.0.1:35045_) [n:127.0.0.1:35045_ c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1708152 INFO  (coreLoadExecutor-3921-thread-1-processing-n:127.0.0.1:35045_) [n:127.0.0.1:35045_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 1708157 INFO  (coreLoadExecutor-3921-thread-1-processing-n:127.0.0.1:35045_) [n:127.0.0.1:35045_ c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 1708158 INFO  (coreLoadExecutor-3921-thread-1-processing-n:127.0.0.1:35045_) [n:127.0.0.1:35045_ c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 1708165 INFO  (coreLoadExecutor-3921-thread-1-processing-n:127.0.0.1:35045_) [n:127.0.0.1:35045_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1708167 INFO  (coreLoadExecutor-3921-thread-1-processing-n:127.0.0.1:35045_) [n:127.0.0.1:35045_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1708169 INFO  (coreLoadExecutor-3921-thread-1-processing-n:127.0.0.1:35045_) [n:127.0.0.1:35045_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1708170 INFO  (coreLoadExecutor-3921-thread-1-processing-n:127.0.0.1:35045_) [n:127.0.0.1:35045_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1708171 INFO  (coreLoadExecutor-3921-thread-1-processing-n:127.0.0.1:35045_) [n:127.0.0.1:35045_ c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1
   [junit4]   2> 1708171 INFO  (coreLoadExecutor-3921-thread-1-processing-n:127.0.0.1:35045_) [n:127.0.0.1:35045_ c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 1708171 INFO  (coreLoadExecutor-3921-thread-1-processing-n:127.0.0.1:35045_) [n:127.0.0.1:35045_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 1708171 INFO  (coreLoadExecutor-3921-thread-1-processing-n:127.0.0.1:35045_) [n:127.0.0.1:35045_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_C8145CEB159DF6BE-001/control-001/cores/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_C8145CEB159DF6BE-001/control-001/cores/collection1/data/]
   [junit4]   2> 1708172 INFO  (coreLoadExecutor-3921-thread-1-processing-n:127.0.0.1:35045_) [n:127.0.0.1:35045_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@5ad4d3f6
   [junit4]   2> 1708172 INFO  (coreLoadExecutor-3921-thread-1-processing-n:127.0.0.1:35045_) [n:127.0.0.1:35045_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_C8145CEB159DF6BE-001/control-001/cores/collection1/data
   [junit4]   2> 1708172 INFO  (coreLoadExecutor-3921-thread-1-processing-n:127.0.0.1:35045_) [n:127.0.0.1:35045_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_C8145CEB159DF6BE-001/control-001/cores/collection1/data/index/
   [junit4]   2> 1708172 WARN  (coreLoadExecutor-3921-thread-1-processing-n:127.0.0.1:35045_) [n:127.0.0.1:35045_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_C8145CEB159DF6BE-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 1708173 INFO  (coreLoadExecutor-3921-thread-1-processing-n:127.0.0.1:35045_) [n:127.0.0.1:35045_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_C8145CEB159DF6BE-001/control-001/cores/collection1/data/index
   [junit4]   2> 1708173 INFO  (coreLoadExecutor-3921-thread-1-processing-n:127.0.0.1:35045_) [n:127.0.0.1:35045_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=44, maxMergeAtOnceExplicit=25, maxMergedSegmentMB=27.8388671875, floorSegmentMB=1.16015625, forceMergeDeletesPctAllowed=2.535092880785077, segmentsPerTier=32.0, maxCFSSegmentSizeMB=0.3173828125, noCFSRatio=1.0
   [junit4]   2> 1708173 INFO  (coreLoadExecutor-3921-thread-1-processing-n:127.0.0.1:35045_) [n:127.0.0.1:35045_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=RawDirectoryWrapper(RAMDirectory@76fbc153 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@ebfa52e),segFN=segments_1,generation=1}
   [junit4]   2> 1708173 INFO  (coreLoadExecutor-3921-thread-1-processing-n:127.0.0.1:35045_) [n:127.0.0.1:35045_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 1708174 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:35045_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_C8145CEB159DF6BE-001/control-001/cores/collection1/data/
   [junit4]   2> 1708174 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:35045_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_C8145CEB159DF6BE-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 1708177 INFO  (coreLoadExecutor-3921-thread-1-processing-n:127.0.0.1:35045_) [n:127.0.0.1:35045_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 1708177 INFO  (coreLoadExecutor-3921-thread-1-processing-n:127.0.0.1:35045_) [n:127.0.0.1:35045_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 1708177 INFO  (coreLoadExecutor-3921-thread-1-processing-n:127.0.0.1:35045_) [n:127.0.0.1:35045_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 1708177 INFO  (coreLoadExecutor-3921-thread-1-processing-n:127.0.0.1:35045_) [n:127.0.0.1:35045_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1708177 INFO  (coreLoadExecutor-3921-thread-1-processing-n:127.0.0.1:35045_) [n:127.0.0.1:35045_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1708177 INFO  (coreLoadExecutor-3921-thread-1-processing-n:127.0.0.1:35045_) [n:127.0.0.1:35045_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 1708178 INFO  (coreLoadExecutor-3921-thread-1-processing-n:127.0.0.1:35045_) [n:127.0.0.1:35045_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1708178 INFO  (coreLoadExecutor-3921-thread-1-processing-n:127.0.0.1:35045_) [n:127.0.0.1:35045_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1708178 INFO  (coreLoadExecutor-3921-thread-1-processing-n:127.0.0.1:35045_) [n:127.0.0.1:35045_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 1708178 WARN  (coreLoadExecutor-3921-thread-1-processing-n:127.0.0.1:35045_) [n:127.0.0.1:35045_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
   [junit4]   2> 1708180 INFO  (coreLoadExecutor-3921-thread-1-processing-n:127.0.0.1:35045_) [n:127.0.0.1:35045_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1708180 INFO  (coreLoadExecutor-3921-thread-1-processing-n:127.0.0.1:35045_) [n:127.0.0.1:35045_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1708181 INFO  (coreLoadExecutor-3921-thread-1-processing-n:127.0.0.1:35045_) [n:127.0.0.1:35045_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1708182 INFO  (coreLoadExecutor-3921-thread-1-processing-n:127.0.0.1:35045_) [n:127.0.0.1:35045_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1708185 INFO  (coreLoadExecutor-3921-thread-1-processing-n:127.0.0.1:35045_) [n:127.0.0.1:35045_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/mbeans,standard,/dump,/update/csv,/sql,/update/json/docs,/admin/luke,/export,/admin/segments,/get,/admin/system,/replication,/admin/properties,/config,/stream,/schema,/admin/plugins,/admin/logging,/update/json,/admin/ping,/admin/threads,/update,/admin/file
   [junit4]   2> 1708185 INFO  (coreLoadExecutor-3921-thread-1-processing-n:127.0.0.1:35045_) [n:127.0.0.1:35045_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 1708185 INFO  (coreLoadExecutor-3921-thread-1-processing-n:127.0.0.1:35045_) [n:127.0.0.1:35045_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1708185 INFO  (coreLoadExecutor-3921-thread-1-processing-n:127.0.0.1:35045_) [n:127.0.0.1:35045_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1708186 INFO  (coreLoadExecutor-3921-thread-1-processing-n:127.0.0.1:35045_) [n:127.0.0.1:35045_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1708186 INFO  (coreLoadExecutor-3921-thread-1-processing-n:127.0.0.1:35045_) [n:127.0.0.1:35045_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1708186 INFO  (coreLoadExecutor-3921-thread-1-processing-n:127.0.0.1:35045_) [n:127.0.0.1:35045_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=10, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.7771397645231398]
   [junit4]   2> 1708186 INFO  (coreLoadExecutor-3921-thread-1-processing-n:127.0.0.1:35045_) [n:127.0.0.1:35045_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=RawDirectoryWrapper(RAMDirectory@76fbc153 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@ebfa52e),segFN=segments_1,generation=1}
   [junit4]   2> 1708186 INFO  (coreLoadExecutor-3921-thread-1-processing-n:127.0.0.1:35045_) [n:127.0.0.1:35045_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 1708187 INFO  (coreLoadExecutor-3921-thread-1-processing-n:127.0.0.1:35045_) [n:127.0.0.1:35045_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@632f1b73[collection1] main]
   [junit4]   2> 1708187 INFO  (coreLoadExecutor-3921-thread-1-processing-n:127.0.0.1:35045_) [n:127.0.0.1:35045_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/control_collection]
   [junit4]   2> 1708188 INFO  (coreLoadExecutor-3921-thread-1-processing-n:127.0.0.1:35045_) [n:127.0.0.1:35045_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/control_collection] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 1708188 INFO  (coreLoadExecutor-3921-thread-1-processing-n:127.0.0.1:35045_) [n:127.0.0.1:35045_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 1708188 INFO  (coreLoadExecutor-3921-thread-1-processing-n:127.0.0.1:35045_) [n:127.0.0.1:35045_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1708188 INFO  (coreLoadExecutor-3921-thread-1-processing-n:127.0.0.1:35045_) [n:127.0.0.1:35045_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 1708188 INFO  (coreLoadExecutor-3921-thread-1-processing-n:127.0.0.1:35045_) [n:127.0.0.1:35045_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1708188 INFO  (coreLoadExecutor-3921-thread-1-processing-n:127.0.0.1:35045_) [n:127.0.0.1:35045_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 1708188 INFO  (coreLoadExecutor-3921-thread-1-processing-n:127.0.0.1:35045_) [n:127.0.0.1:35045_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1708188 INFO  (coreLoadExecutor-3921-thread-1-processing-n:127.0.0.1:35045_) [n:127.0.0.1:35045_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 1708188 INFO  (coreLoadExecutor-3921-thread-1-processing-n:127.0.0.1:35045_) [n:127.0.0.1:35045_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1708189 INFO  (searcherExecutor-3922-thread-1-processing-n:127.0.0.1:35045_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:35045_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@632f1b73[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1708189 INFO  (coreLoadExecutor-3921-thread-1-processing-n:127.0.0.1:35045_) [n:127.0.0.1:35045_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 1708189 INFO  (coreLoadExecutor-3921-thread-1-processing-n:127.0.0.1:35045_) [n:127.0.0.1:35045_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 1708189 INFO  (coreLoadExecutor-3921-thread-1-processing-n:127.0.0.1:35045_) [n:127.0.0.1:35045_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 1708189 INFO  (coreLoadExecutor-3921-thread-1-processing-n:127.0.0.1:35045_) [n:127.0.0.1:35045_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1533346799881814016
   [junit4]   2> 1708191 INFO  (coreLoadExecutor-3921-thread-1-processing-n:127.0.0.1:35045_) [n:127.0.0.1:35045_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 1.0ms to seed version buckets with highest version 1533346799881814016
   [junit4]   2> 1708191 INFO  (coreLoadExecutor-3921-thread-1-processing-n:127.0.0.1:35045_) [n:127.0.0.1:35045_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 1708192 INFO  (coreLoadExecutor-3921-thread-1-processing-n:127.0.0.1:35045_) [n:127.0.0.1:35045_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 1708192 INFO  (coreZkRegister-3915-thread-1-processing-n:127.0.0.1:35045_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:35045_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:35045 collection:control_collection shard:shard1
   [junit4]   2> 1708195 INFO  (coreZkRegister-3915-thread-1-processing-n:127.0.0.1:35045_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:35045_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 1708198 INFO  (coreZkRegister-3915-thread-1-processing-n:127.0.0.1:35045_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:35045_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/95834174887100422-core_node1-n_0000000000
   [junit4]   2> 1708198 INFO  (coreZkRegister-3915-thread-1-processing-n:127.0.0.1:35045_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:35045_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard=shard1 and weAreReplacement=false and leaderVoteWait=180000
   [junit4]   2> 1708199 INFO  (coreZkRegister-3915-thread-1-processing-n:127.0.0.1:35045_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:35045_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1708199 INFO  (coreZkRegister-3915-thread-1-processing-n:127.0.0.1:35045_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:35045_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1708199 INFO  (coreZkRegister-3915-thread-1-processing-n:127.0.0.1:35045_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:35045_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:35045/collection1/
   [junit4]   2> 1708199 INFO  (coreZkRegister-3915-thread-1-processing-n:127.0.0.1:35045_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:35045_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 1708199 INFO  (coreZkRegister-3915-thread-1-processing-n:127.0.0.1:35045_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:35045_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:35045/collection1/ has no replicas
   [junit4]   2> 1708200 INFO  (OverseerStateUpdate-95834174887100422-127.0.0.1:35045_-n_0000000000) [n:127.0.0.1:35045_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 1708200 INFO  (coreZkRegister-3915-thread-1-processing-n:127.0.0.1:35045_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:35045_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 1708202 INFO  (coreZkRegister-3915-thread-1-processing-n:127.0.0.1:35045_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:35045_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/control_collection/leaders/shard1/leader after winning as /collections/control_collection/leader_elect/shard1/election/95834174887100422-core_node1-n_0000000000
   [junit4]   2> 1708203 INFO  (coreZkRegister-3915-thread-1-processing-n:127.0.0.1:35045_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:35045_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:35045/collection1/ shard1
   [junit4]   2> 1708204 INFO  (OverseerStateUpdate-95834174887100422-127.0.0.1:35045_-n_0000000000) [n:127.0.0.1:35045_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:35045",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 1
   [junit4]   2> 1708306 INFO  (zkCallback-1212-thread-1-processing-n:127.0.0.1:35045_) [n:127.0.0.1:35045_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1708354 INFO  (coreZkRegister-3915-thread-1-processing-n:127.0.0.1:35045_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:35045_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:35045/collection1/ and leader is http://127.0.0.1:35045/collection1/
   [junit4]   2> 1708354 INFO  (coreZkRegister-3915-thread-1-processing-n:127.0.0.1:35045_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:35045_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:35045
   [junit4]   2> 1708354 INFO  (coreZkRegister-3915-thread-1-processing-n:127.0.0.1:35045_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:35045_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1708354 INFO  (coreZkRegister-3915-thread-1-processing-n:127.0.0.1:35045_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:35045_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 1708354 INFO  (coreZkRegister-3915-thread-1-processing-n:127.0.0.1:35045_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:35045_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1708356 INFO  (OverseerStateUpdate-95834174887100422-127.0.0.1:35045_-n_0000000000) [n:127.0.0.1:35045_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:35045",
   [junit4]   2>   "node_name":"127.0.0.1:35045_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"} current state version: 2
   [junit4]   2> 1708356 INFO  (OverseerStateUpdate-95834174887100422-127.0.0.1:35045_-n_0000000000) [n:127.0.0.1:35045_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:35045",
   [junit4]   2>   "node_name":"127.0.0.1:35045_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 1708458 INFO  (zkCallback-1212-thread-1-processing-n:127.0.0.1:35045_) [n:127.0.0.1:35045_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1708544 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.s.i.CloudSolrClient Final constructed zkHost string: 127.0.0.1:56141/solr
   [junit4]   2> 1708545 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1708545 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1708547 INFO  (zkCallback-1214-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6459d11a name:ZooKeeperConnection Watcher:127.0.0.1:56141/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1708547 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1708547 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1708547 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1708549 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1708549 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 1708638 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_C8145CEB159DF6BE-001/shard-1-001/cores/collection1
   [junit4]   2> 1708639 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_C8145CEB159DF6BE-001/shard-1-001
   [junit4]   2> 1708639 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.e.j.s.Server jetty-9.3.8.v20160314
   [junit4]   2> 1708641 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1bb09465{/,null,AVAILABLE}
   [junit4]   2> 1708641 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.e.j.s.ServerConnector Started ServerConnector@1c4c9441{HTTP/1.1,[http/1.1]}{127.0.0.1:59307}
   [junit4]   2> 1708642 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.e.j.s.Server Started @1711788ms
   [junit4]   2> 1708642 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_C8145CEB159DF6BE-001/tempDir-001/jetty1, solrconfig=solrconfig.xml, hostContext=/, hostPort=59307, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_C8145CEB159DF6BE-001/shard-1-001/cores}
   [junit4]   2> 1708642 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@73d16e93
   [junit4]   2> 1708642 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_C8145CEB159DF6BE-001/shard-1-001'
   [junit4]   2> 1708642 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1708642 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1708642 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1708643 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1708644 INFO  (zkCallback-1215-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@32258027 name:ZooKeeperConnection Watcher:127.0.0.1:56141/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1708644 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1708644 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1708645 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1708645 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_C8145CEB159DF6BE-001/shard-1-001/solr.xml
   [junit4]   2> 1708649 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_C8145CEB159DF6BE-001/shard-1-001/cores
   [junit4]   2> 1708650 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.CoreContainer New CoreContainer 1975698588
   [junit4]   2> 1708650 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_C8145CEB159DF6BE-001/shard-1-001]
   [junit4]   2> 1708650 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_C8145CEB159DF6BE-001/shard-1-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_C8145CEB159DF6BE-001/shard-1-001/lib
   [junit4]   2> 1708650 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1708651 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1708651 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1708651 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1708651 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:56141/solr
   [junit4]   2> 1708652 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1708652 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1708652 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1708653 INFO  (zkCallback-1218-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@567f1878 name:ZooKeeperConnection Watcher:127.0.0.1:56141 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1708653 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1708653 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1708654 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [n:127.0.0.1:59307_    ] o.a.s.c.ZkController Added new OnReconnect listener org.apache.solr.cloud.ZkController$$Lambda$47/1525438528@794a49e1
   [junit4]   2> 1708654 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [n:127.0.0.1:59307_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1708655 INFO  (zkCallback-1219-thread-1-processing-n:127.0.0.1:59307_) [n:127.0.0.1:59307_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2bf473de name:ZooKeeperConnection Watcher:127.0.0.1:56141/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1708655 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [n:127.0.0.1:59307_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1708661 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [n:127.0.0.1:59307_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1708662 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [n:127.0.0.1:59307_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1708664 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [n:127.0.0.1:59307_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1708664 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [n:127.0.0.1:59307_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/95834174887100426-127.0.0.1:59307_-n_0000000001
   [junit4]   2> 1708664 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [n:127.0.0.1:59307_    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/95834174887100422-127.0.0.1:35045_-n_0000000000 to know if I could be the leader
   [junit4]   2> 1708665 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[C8145CEB159DF6BE]) [n:127.0.0.1:59307_    ] o.a.s.c.ZkController Publish node=127.0.0.1:59307_ as DOWN
   [junit4]   2> 1708666 INFO  (OverseerStateUpdate-95834174887100422-127.0.0.1:35045_-n_0000000000) [n:127.

[...truncated too long message...]

g-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-collective-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-event-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-exception-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-journal-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-normalization-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-operational-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-referral-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-schema-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-subtree-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-trigger-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-jdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-jdbm1-2.0.0-M2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-kerberos-codec-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-ldif-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-mavibot-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-protocol-kerberos-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-protocol-ldap-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-protocol-shared-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-xdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/api-all-1.0.0-M20.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/bcprov-jdk15-1.45.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/cglib-nodep-2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/commons-collections-3.2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/commons-math3-3.4.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/easymock-3.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/ehcache-core-2.4.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/hadoop-common-2.6.0-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/hadoop-hdfs-2.6.0-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/hadoop-minikdc-2.6.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jackson-annotations-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jackson-databind-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jersey-core-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jersey-server-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jetty-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jetty-util-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/mina-core-2.0.0-M5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/mockito-core-1.9.5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/objenesis-1.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/presto-parser-0.122.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/slice-0.10.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/analysis/icu/lucene-analyzers-icu-6.1.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/contrib/solr-analysis-extras/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/queryparser/classes/test:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/contrib/analysis-extras/lib/icu4j-56.1.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-launcher.jar:/x1/jenkins/.ant/lib/ivy-2.3.0.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-antlr.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-regexp.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-resolver.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-net.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jmf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-oro.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-xalan2.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-logging.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jsch.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit4.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-javamail.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jai.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jdepend.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bsf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-swing.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-netrexx.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bcel.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-testutil.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-log4j.jar:/x1/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.8/lib/tools.jar:/x1/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.3.4.jar com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/temp/junit4-J0-20160503_214229_802.events @/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/temp/junit4-J0-20160503_214229_802.suites -stdin
   [junit4] ERROR: JVM J0 ended with an exception: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1501)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:121)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:955)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:952)
   [junit4] 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4] 	at java.lang.Thread.run(Thread.java:745)

BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:747: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:684: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/common-build.xml:529: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/common-build.xml:1457: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/common-build.xml:1014: At least one slave process threw an exception, first: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.

Total time: 272 minutes 7 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
No prior successful build to compare, so performing full copy of artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any




[JENKINS] Lucene-Solr-NightlyTests-6.x - Build # 54 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-6.x/54/

All tests passed

Build Log:
[...truncated 12524 lines...]
   [junit4] JVM J2: stdout was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/temp/junit4-J2-20160501_230636_014.sysout
   [junit4] >>> JVM J2 emitted unexpected output (verbatim) ----
   [junit4] java.lang.OutOfMemoryError: GC overhead limit exceeded
   [junit4] Dumping heap to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/heapdumps/java_pid11594.hprof ...
   [junit4] Heap dump file created [615402481 bytes in 2.732 secs]
   [junit4] <<< JVM J2: EOF ----

   [junit4] JVM J2: stderr was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/temp/junit4-J2-20160501_230636_014.syserr
   [junit4] >>> JVM J2 emitted unexpected output (verbatim) ----
   [junit4] 
   [junit4] Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "SessionTracker"
   [junit4] WARN: Unhandled exception in event serialization. -> java.lang.OutOfMemoryError: GC overhead limit exceeded
   [junit4] <<< JVM J2: EOF ----

[...truncated 339 lines...]
   [junit4] ERROR: JVM J2 ended with an exception, command line: /x1/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.8/jre/bin/java -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/heapdumps -ea -esa -Dtests.prefix=tests -Dtests.seed=DFDAC0F3EAFBB7BB -Xmx512M -Dtests.iters= -Dtests.verbose=false -Dtests.infostream=false -Dtests.codec=random -Dtests.postingsformat=random -Dtests.docvaluesformat=random -Dtests.locale=random -Dtests.timezone=random -Dtests.directory=random -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.luceneMatchVersion=6.1.0 -Dtests.cleanthreads=perClass -Djava.util.logging.config.file=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/tools/junit4/logging.properties -Dtests.nightly=true -Dtests.weekly=false -Dtests.monster=false -Dtests.slow=true -Dtests.asserts=true -Dtests.multiplier=2 -DtempDir=./temp -Djava.io.tmpdir=./temp -Djunit4.tempDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/temp -Dcommon.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene -Dclover.db.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/clover/db -Djava.security.policy=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/tools/junit4/solr-tests.policy -Dtests.LUCENE_VERSION=6.1.0 -Djetty.testMode=1 -Djetty.insecurerandom=1 -Dsolr.directoryFactory=org.apache.solr.core.MockDirectoryFactory -Djava.awt.headless=true -Djdk.map.althashing.threshold=0 -Djunit4.childvm.cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2 -Djunit4.childvm.id=2 -Djunit4.childvm.count=3 -Dtests.leaveTemporary=false -Dtests.filterstacks=true -Djava.security.manager=org.apache.lucene.util.TestSecurityManager -Dfile.encoding=ISO-8859-1 -classpath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/classes/test:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-test-framework/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/test-framework/lib/junit4-ant-2.3.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/test-framework/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/codecs/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-solrj/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/analysis/common/lucene-analyzers-common-6.1.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/analysis/kuromoji/lucene-analyzers-kuromoji-6.1.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/analysis/phonetic/lucene-analyzers-phonetic-6.1.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/codecs/lucene-codecs-6.1.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/backward-codecs/lucene-backward-codecs-6.1.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/highlighter/lucene-highlighter-6.1.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/memory/lucene-memory-6.1.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/misc/lucene-misc-6.1.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/spatial-extras/lucene-spatial-extras-6.1.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/expressions/lucene-expressions-6.1.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/suggest/lucene-suggest-6.1.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/grouping/lucene-grouping-6.1.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/queries/lucene-queries-6.1.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/queryparser/lucene-queryparser-6.1.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/join/lucene-join-6.1.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/sandbox/lucene-sandbox-6.1.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/classification/lucene-classification-6.1.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/lib/antlr4-runtime-4.5.1-1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/lib/asm-5.0.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/lib/asm-commons-5.0.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/lib/caffeine-1.0.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/lib/commons-cli-1.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/lib/commons-codec-1.10.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/lib/commons-collections-3.2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/lib/commons-configuration-1.6.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/lib/commons-exec-1.3.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/lib/commons-fileupload-1.2.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/lib/commons-lang-2.6.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/lib/dom4j-1.6.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/lib/guava-14.0.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/lib/hadoop-annotations-2.6.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/lib/hadoop-auth-2.6.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/lib/hadoop-common-2.6.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/lib/hadoop-hdfs-2.6.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/lib/hppc-0.7.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/lib/htrace-core-3.0.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/lib/jackson-core-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/lib/jackson-dataformat-smile-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/lib/jcl-over-slf4j-1.7.7.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/lib/joda-time-2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/lib/log4j-1.2.17.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/lib/org.restlet-2.3.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/lib/org.restlet.ext.servlet-2.3.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/lib/presto-parser-0.122.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/lib/protobuf-java-2.5.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/lib/slf4j-log4j12-1.7.7.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/lib/slice-0.10.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/lib/spatial4j-0.6.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/lib/t-digest-3.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/solrj/lib/commons-io-2.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/solrj/lib/httpclient-4.4.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/solrj/lib/httpcore-4.4.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/solrj/lib/httpmime-4.4.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/solrj/lib/jcl-over-slf4j-1.7.7.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/solrj/lib/noggit-0.6.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/solrj/lib/slf4j-api-1.7.7.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/solrj/lib/stax2-api-3.1.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/solrj/lib/woodstox-core-asl-4.4.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/solrj/lib/zookeeper-3.4.6.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/server/lib/javax.servlet-api-3.1.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/server/lib/jetty-continuation-9.3.8.v20160314.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/server/lib/jetty-deploy-9.3.8.v20160314.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/server/lib/jetty-http-9.3.8.v20160314.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/server/lib/jetty-io-9.3.8.v20160314.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/server/lib/jetty-jmx-9.3.8.v20160314.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/server/lib/jetty-rewrite-9.3.8.v20160314.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/server/lib/jetty-security-9.3.8.v20160314.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/server/lib/jetty-server-9.3.8.v20160314.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/server/lib/jetty-servlet-9.3.8.v20160314.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/server/lib/jetty-servlets-9.3.8.v20160314.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/server/lib/jetty-util-9.3.8.v20160314.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/server/lib/jetty-webapp-9.3.8.v20160314.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/server/lib/jetty-xml-9.3.8.v20160314.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/example/example-DIH/solr/db/lib/derby-10.9.1.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/example/example-DIH/solr/db/lib/hsqldb-1.8.0.10.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/test-framework/lib/junit-4.10.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/test-framework/lib/randomizedtesting-runner-2.3.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/antlr-2.7.7.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/antlr4-runtime-4.5.1-1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-core-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-core-api-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-core-avl-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-core-shared-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-i18n-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptor-kerberos-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-admin-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-authn-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-authz-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-changelog-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-collective-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-event-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-exception-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-journal-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-normalization-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-operational-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-referral-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-schema-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-subtree-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-trigger-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-jdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-jdbm1-2.0.0-M2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-kerberos-codec-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-ldif-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-mavibot-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-protocol-kerberos-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-protocol-ldap-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-protocol-shared-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-xdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/api-all-1.0.0-M20.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/bcprov-jdk15-1.45.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/cglib-nodep-2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/commons-collections-3.2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/commons-math3-3.4.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/easymock-3.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/ehcache-core-2.4.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/hadoop-common-2.6.0-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/hadoop-hdfs-2.6.0-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/hadoop-minikdc-2.6.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jackson-annotations-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jackson-databind-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jersey-core-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jersey-server-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jetty-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jetty-util-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/mina-core-2.0.0-M5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/mockito-core-1.9.5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/objenesis-1.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/presto-parser-0.122.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/slice-0.10.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/analysis/icu/lucene-analyzers-icu-6.1.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/contrib/solr-analysis-extras/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/queryparser/classes/test:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/contrib/analysis-extras/lib/icu4j-56.1.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-launcher.jar:/x1/jenkins/.ant/lib/ivy-2.3.0.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-antlr.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-regexp.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-resolver.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-net.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jmf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-oro.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-xalan2.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-logging.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jsch.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit4.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-javamail.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jai.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jdepend.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bsf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-swing.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-netrexx.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bcel.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-testutil.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-log4j.jar:/x1/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.8/lib/tools.jar:/x1/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.3.4.jar com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/temp/junit4-J2-20160501_230636_014.events @/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/temp/junit4-J2-20160501_230636_014.suites -stdin
   [junit4] ERROR: JVM J2 ended with an exception: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1501)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:121)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:955)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:952)
   [junit4] 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4] 	at java.lang.Thread.run(Thread.java:745)

BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:747: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:684: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/common-build.xml:529: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/common-build.xml:1457: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/common-build.xml:1014: At least one slave process threw an exception, first: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.

Total time: 211 minutes 12 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
No prior successful build to compare, so performing full copy of artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any




[JENKINS] Lucene-Solr-NightlyTests-6.x - Build # 53 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-6.x/53/

2 tests failed.
FAILED:  org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.test

Error Message:
Timeout occured while waiting response from server at: http://127.0.0.1:35756/q_s/nl

Stack Trace:
org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:35756/q_s/nl
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:601)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:259)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:248)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.makeRequest(CollectionsAPIDistributedZkTest.java:399)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testErrorHandling(CollectionsAPIDistributedZkTest.java:457)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test(CollectionsAPIDistributedZkTest.java:179)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.net.SocketTimeoutException: Read timed out
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
	at java.net.SocketInputStream.read(SocketInputStream.java:170)
	at java.net.SocketInputStream.read(SocketInputStream.java:141)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:160)
	at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:84)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:273)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
	at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
	at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)
	at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)
	at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197)
	at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)
	at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)
	at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685)
	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487)
	at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:495)
	... 47 more


FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Timeout occured while waiting response from server at: http://127.0.0.1:59625/htp

Stack Trace:
org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:59625/htp
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:601)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:259)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:248)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.makeRequest(CollectionsAPIDistributedZkTest.java:399)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testErrorHandling(CollectionsAPIDistributedZkTest.java:457)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test(CollectionsAPIDistributedZkTest.java:179)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.net.SocketTimeoutException: Read timed out
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
	at java.net.SocketInputStream.read(SocketInputStream.java:170)
	at java.net.SocketInputStream.read(SocketInputStream.java:141)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:160)
	at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:84)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:273)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
	at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
	at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)
	at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)
	at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197)
	at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)
	at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)
	at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685)
	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487)
	at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:495)
	... 47 more




Build Log:
[...truncated 11549 lines...]
   [junit4] Suite: org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_54A59967B5CDEEA1-001/init-core-data-001
   [junit4]   2> 1053049 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[54A59967B5CDEEA1]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false)
   [junit4]   2> 1053050 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[54A59967B5CDEEA1]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /q_s/nl
   [junit4]   1> Formatting using clusterid: testClusterID
   [junit4]   2> 1053101 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[54A59967B5CDEEA1]-worker) [    ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
   [junit4]   2> 1053107 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[54A59967B5CDEEA1]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 1053109 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[54A59967B5CDEEA1]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 1053118 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[54A59967B5CDEEA1]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/hdfs to ./temp/Jetty_localhost_36876_hdfs____.mh7mgy/webapp
   [junit4]   2> 1053198 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[54A59967B5CDEEA1]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 1053535 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[54A59967B5CDEEA1]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:36876
   [junit4]   2> 1053614 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[54A59967B5CDEEA1]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 1053615 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[54A59967B5CDEEA1]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 1053627 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[54A59967B5CDEEA1]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_40882_datanode____.622wv7/webapp
   [junit4]   2> 1053712 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[54A59967B5CDEEA1]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 1054109 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[54A59967B5CDEEA1]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:40882
   [junit4]   2> 1054177 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[54A59967B5CDEEA1]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 1054178 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[54A59967B5CDEEA1]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 1054193 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[54A59967B5CDEEA1]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_41105_datanode____5d7a2o/webapp
   [junit4]   2> 1054298 INFO  (IPC Server handler 0 on 46442) [    ] BlockStateChange BLOCK* processReport: from storage DS-196ac04a-ca8c-409d-9c07-cb11eef0f31b node DatanodeRegistration(127.0.0.1, datanodeUuid=81d99c3d-719d-4d6b-9066-1db4acdfc845, infoPort=40882, ipcPort=54195, storageInfo=lv=-56;cid=testClusterID;nsid=1522856014;c=0), blocks: 0, hasStaleStorages: true, processing time: 1 msecs
   [junit4]   2> 1054298 INFO  (IPC Server handler 0 on 46442) [    ] BlockStateChange BLOCK* processReport: from storage DS-0740bc36-0b7e-49cc-8a35-92cc8235e0e4 node DatanodeRegistration(127.0.0.1, datanodeUuid=81d99c3d-719d-4d6b-9066-1db4acdfc845, infoPort=40882, ipcPort=54195, storageInfo=lv=-56;cid=testClusterID;nsid=1522856014;c=0), blocks: 0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 1054318 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[54A59967B5CDEEA1]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 1054756 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[54A59967B5CDEEA1]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:41105
   [junit4]   2> 1054821 INFO  (IPC Server handler 1 on 46442) [    ] BlockStateChange BLOCK* processReport: from storage DS-3ccc41be-04d3-460e-987d-c074591f2af0 node DatanodeRegistration(127.0.0.1, datanodeUuid=2e289dbb-da58-4768-a06e-ba3f20b8f7ef, infoPort=41105, ipcPort=35063, storageInfo=lv=-56;cid=testClusterID;nsid=1522856014;c=0), blocks: 0, hasStaleStorages: true, processing time: 1 msecs
   [junit4]   2> 1054821 INFO  (IPC Server handler 1 on 46442) [    ] BlockStateChange BLOCK* processReport: from storage DS-5d914457-994b-4549-937c-f5c65ba8391b node DatanodeRegistration(127.0.0.1, datanodeUuid=2e289dbb-da58-4768-a06e-ba3f20b8f7ef, infoPort=41105, ipcPort=35063, storageInfo=lv=-56;cid=testClusterID;nsid=1522856014;c=0), blocks: 0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 1054923 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 1054923 INFO  (Thread-7579) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1054923 INFO  (Thread-7579) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 1055023 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [    ] o.a.s.c.ZkTestServer start zk server on port:46819
   [junit4]   2> 1055023 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1055024 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1055026 INFO  (zkCallback-1657-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5d37ab42 name:ZooKeeperConnection Watcher:127.0.0.1:46819 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1055026 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1055026 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1055026 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 1055027 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1055028 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1055029 INFO  (zkCallback-1658-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@751baaf6 name:ZooKeeperConnection Watcher:127.0.0.1:46819/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1055029 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1055029 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1055029 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 1055031 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 1055032 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 1055033 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 1055034 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 1055034 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 1055036 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 1055036 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 1055037 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1055037 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1055038 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 1055038 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 1055039 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 1055039 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 1055039 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 1055039 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 1055040 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 1055040 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 1055041 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1055041 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1055043 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1055043 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1055045 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 1055045 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 1055046 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 1055046 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 1055049 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1055049 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1055050 INFO  (zkCallback-1659-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3f46d1c8 name:ZooKeeperConnection Watcher:127.0.0.1:46819 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1055050 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1055050 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1055050 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 1055051 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1055051 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1055051 INFO  (zkCallback-1660-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@27f3e05d name:ZooKeeperConnection Watcher:127.0.0.1:46819/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1055051 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1055051 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1055052 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf2/solrconfig.xml
   [junit4]   2> 1055052 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/solrconfig.xml
   [junit4]   2> 1055053 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf2/schema.xml
   [junit4]   2> 1055053 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/schema.xml
   [junit4]   2> 1055054 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf2/enumsConfig.xml
   [junit4]   2> 1055054 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/enumsConfig.xml
   [junit4]   2> 1055055 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf2/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1055055 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1055056 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf2/stopwords.txt
   [junit4]   2> 1055056 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/stopwords.txt
   [junit4]   2> 1055057 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf2/protwords.txt
   [junit4]   2> 1055057 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/protwords.txt
   [junit4]   2> 1055057 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf2/currency.xml
   [junit4]   2> 1055057 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/currency.xml
   [junit4]   2> 1055058 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf2/open-exchange-rates.json
   [junit4]   2> 1055058 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/open-exchange-rates.json
   [junit4]   2> 1055059 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf2/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1055059 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1055060 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf2/old_synonyms.txt
   [junit4]   2> 1055060 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/old_synonyms.txt
   [junit4]   2> 1055061 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf2/synonyms.txt
   [junit4]   2> 1055061 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/synonyms.txt
   [junit4]   2> 1055062 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/elevate.xml to /configs/conf2/elevate.xml
   [junit4]   2> 1055062 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/elevate.xml
   [junit4]   2> 1055120 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_54A59967B5CDEEA1-001/control-001/cores/collection1
   [junit4]   2> 1055121 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [    ] o.e.j.s.Server jetty-9.3.8.v20160314
   [junit4]   2> 1055122 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@703be2b6{/q_s/nl,null,AVAILABLE}
   [junit4]   2> 1055122 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [    ] o.e.j.s.ServerConnector Started ServerConnector@23ba2398{HTTP/1.1,[http/1.1]}{127.0.0.1:56494}
   [junit4]   2> 1055122 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [    ] o.e.j.s.Server Started @1057204ms
   [junit4]   2> 1055123 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=hdfs://localhost:46442/hdfs__localhost_46442__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-6.x_solr_build_solr-core_test_J1_temp_solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_54A59967B5CDEEA1-001_tempDir-002_control_data, hostContext=/q_s/nl, hostPort=56494, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_54A59967B5CDEEA1-001/control-001/cores}
   [junit4]   2> 1055123 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@73d16e93
   [junit4]   2> 1055123 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_54A59967B5CDEEA1-001/control-001'
   [junit4]   2> 1055123 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1055123 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1055123 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1055123 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1055125 INFO  (zkCallback-1661-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3131617c name:ZooKeeperConnection Watcher:127.0.0.1:46819/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1055125 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1055125 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1055125 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1055126 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_54A59967B5CDEEA1-001/control-001/solr.xml
   [junit4]   2> 1055129 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_54A59967B5CDEEA1-001/control-001/cores
   [junit4]   2> 1055129 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [    ] o.a.s.c.CoreContainer New CoreContainer 1310770332
   [junit4]   2> 1055129 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_54A59967B5CDEEA1-001/control-001]
   [junit4]   2> 1055129 WARN  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_54A59967B5CDEEA1-001/control-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_54A59967B5CDEEA1-001/control-001/lib
   [junit4]   2> 1055129 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1055130 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1055130 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1055130 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1055131 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46819/solr
   [junit4]   2> 1055131 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1055131 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1055131 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1055133 INFO  (zkCallback-1664-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4025df80 name:ZooKeeperConnection Watcher:127.0.0.1:46819 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1055134 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1055134 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1055135 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [n:127.0.0.1:56494_q_s%2Fnl    ] o.a.s.c.ZkController Added new OnReconnect listener org.apache.solr.cloud.ZkController$$Lambda$32/702459987@42918664
   [junit4]   2> 1055135 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [n:127.0.0.1:56494_q_s%2Fnl    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1055136 INFO  (zkCallback-1665-thread-1-processing-n:127.0.0.1:56494_q_s%2Fnl) [n:127.0.0.1:56494_q_s%2Fnl    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@68ba7591 name:ZooKeeperConnection Watcher:127.0.0.1:46819/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1055137 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [n:127.0.0.1:56494_q_s%2Fnl    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1055138 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [n:127.0.0.1:56494_q_s%2Fnl    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 1055138 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [n:127.0.0.1:56494_q_s%2Fnl    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 1055140 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [n:127.0.0.1:56494_q_s%2Fnl    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 1055141 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [n:127.0.0.1:56494_q_s%2Fnl    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 1055142 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [n:127.0.0.1:56494_q_s%2Fnl    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 1055143 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [n:127.0.0.1:56494_q_s%2Fnl    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 1055143 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [n:127.0.0.1:56494_q_s%2Fnl    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 1055144 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [n:127.0.0.1:56494_q_s%2Fnl    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 1055145 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [n:127.0.0.1:56494_q_s%2Fnl    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 1055145 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [n:127.0.0.1:56494_q_s%2Fnl    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1055146 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [n:127.0.0.1:56494_q_s%2Fnl    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (0)
   [junit4]   2> 1055146 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [n:127.0.0.1:56494_q_s%2Fnl    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 1055147 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [n:127.0.0.1:56494_q_s%2Fnl    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 1055148 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [n:127.0.0.1:56494_q_s%2Fnl    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1055148 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [n:127.0.0.1:56494_q_s%2Fnl    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/95817762279849990-127.0.0.1:56494_q_s%2Fnl-n_0000000000
   [junit4]   2> 1055148 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [n:127.0.0.1:56494_q_s%2Fnl    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:56494_q_s%2Fnl
   [junit4]   2> 1055148 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [n:127.0.0.1:56494_q_s%2Fnl    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 1055149 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [n:127.0.0.1:56494_q_s%2Fnl    ] o.a.s.c.Overseer Overseer (id=95817762279849990-127.0.0.1:56494_q_s%2Fnl-n_0000000000) starting
   [junit4]   2> 1055150 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [n:127.0.0.1:56494_q_s%2Fnl    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 1055153 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [n:127.0.0.1:56494_q_s%2Fnl    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 1055153 INFO  (OverseerCollectionConfigSetProcessor-95817762279849990-127.0.0.1:56494_q_s%2Fnl-n_0000000000) [n:127.0.0.1:56494_q_s%2Fnl    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 1055153 INFO  (OverseerStateUpdate-95817762279849990-127.0.0.1:56494_q_s%2Fnl-n_0000000000) [n:127.0.0.1:56494_q_s%2Fnl    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 1055154 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [n:127.0.0.1:56494_q_s%2Fnl    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:56494_q_s%2Fnl
   [junit4]   2> 1055154 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [n:127.0.0.1:56494_q_s%2Fnl    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:56494_q_s%2Fnl
   [junit4]   2> 1055155 INFO  (OverseerStateUpdate-95817762279849990-127.0.0.1:56494_q_s%2Fnl-n_0000000000) [n:127.0.0.1:56494_q_s%2Fnl    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (0)
   [junit4]   2> 1055155 INFO  (zkCallback-1665-thread-1-processing-n:127.0.0.1:56494_q_s%2Fnl) [n:127.0.0.1:56494_q_s%2Fnl    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [0])
   [junit4]   2> 1055155 INFO  (zkCallback-1665-thread-1-processing-n:127.0.0.1:56494_q_s%2Fnl) [n:127.0.0.1:56494_q_s%2Fnl    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1055174 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [n:127.0.0.1:56494_q_s%2Fnl    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1055174 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [n:127.0.0.1:56494_q_s%2Fnl    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1055174 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [n:127.0.0.1:56494_q_s%2Fnl    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_54A59967B5CDEEA1-001/control-001/cores
   [junit4]   2> 1055175 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [n:127.0.0.1:56494_q_s%2Fnl    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, loadOnStartup=true, configSetProperties=configsetprops.json, collection=control_collection, coreNodeName=, dataDir=data/, shard=}
   [junit4]   2> 1055175 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [n:127.0.0.1:56494_q_s%2Fnl    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_54A59967B5CDEEA1-001/control-001/cores/collection1
   [junit4]   2> 1055175 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [n:127.0.0.1:56494_q_s%2Fnl    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 1055175 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [n:127.0.0.1:56494_q_s%2Fnl    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1
   [junit4]   2> 1055175 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[54A59967B5CDEEA1]) [n:127.0.0.1:56494_q_s%2Fnl    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1055176 INFO  (coreLoadExecutor-4011-thread-1-processing-n:127.0.0.1:56494_q_s%2Fnl) [n:127.0.0.1:56494_q_s%2Fnl c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 1055176 INFO  (coreLoadExecutor-4011-thread-1-processing-n:127.0.0.1:56494_q_s%2Fnl) [n:127.0.0.1:56494_q_s%2Fnl c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1055176 INFO  (coreLoadExecutor-4011-thread-1-processing-n:127.0.0.1:56494_q_s%2Fnl) [n:127.0.0.1:56494_q_s%2Fnl c:control_collection   x:collection1] o.a.s.c.ZkController Collection control_collection not visible yet, but flagging it so a watch is registered when it becomes visible
   [junit4]   2> 1055176 INFO  (coreLoadExecutor-4011-thread-1-processing-n:127.0.0.1:56494_q_s%2Fnl) [n:127.0.0.1:56494_q_s%2Fnl c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader addZkWatch [control_collection]
   [junit4]   2> 1055176 INFO  (OverseerStateUpdate-95817762279849990-127.0.0.1:56494_q_s%2Fnl-n_0000000000) [n:127.0.0.1:56494_q_s%2Fnl    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:56494/q_s/nl",
   [junit4]   2>   "node_name":"127.0.0.1:56494_q_s%2Fnl",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 1055177 INFO  (OverseerStateUpdate-95817762279849990-127.0.0.1:56494_q_s%2Fnl-n_0000000000) [n:127.0.0.1:56494_q_s%2Fnl    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:56494/q_s/nl",
   [junit4]   2>   "node_name":"127.0.0.1:56494_q_s%2Fnl",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 1055177 INFO  (coreLoadExecutor-4011-thread-1-processing-n:127.0.0.1:56494_q_s%2Fnl) [n:127.0.0.1:56494_q_s%2Fnl c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Deleting data for [control_collection]
   [junit4]   2> 1055177 INFO  (OverseerStateUpdate-95817762279849990-127.0.0.1:56494_q_s%2Fnl-n_0000000000) [n:127.0.0.1:56494_q_s%2Fnl    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 1055177 INFO  (coreLoadExecutor-4011-thread-1-processing-n:127.0.0.1:56494_q_s%2Fnl) [n:127.0.0.1:56494_q_s%2Fnl c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 1055177 INFO  (OverseerStateUpdate-95817762279849990-127.0.0.1:56494_q_s%2Fnl-n_0000000000) [n:127.0.0.1:56494_q_s%2Fnl    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 1055178 INFO  (zkCallback-1665-thread-1-processing-n:127.0.0.1:56494_q_s%2Fnl) [n:127.0.0.1:56494_q_s%2Fnl    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1056178 INFO  (coreLoadExecutor-4011-thread-1-processing-n:127.0.0.1:56494_q_s%2Fnl) [n:127.0.0.1:56494_q_s%2Fnl c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1056178 INFO  (coreLoadExecutor-4011-thread-1-processing-n:127.0.0.1:56494_q_s%2Fnl) [n:127.0.0.1:56494_q_s%2Fnl c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 1056179 INFO  (coreLoadExecutor-4011-thread-1-processing-n:127.0.0.1:56494_q_s%2Fnl) [n:127.0.0.1:56494_q_s%2Fnl c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 1056179 INFO  (coreLoadExecutor-4011-thread-1-processing-n:127.0.0.1:56494_q_s%2Fnl) [n:127.0.0.1:56494_q_s%2Fnl c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/control_collection]
   [junit4]   2> 1056179 INFO  (coreLoadExecutor-4011-thread-1-processing-n:127.0.0.1:56494_q_s%2Fnl) [n:127.0.0.1:56494_q_s%2Fnl c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/control_collection] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 1056179 INFO  (coreLoadExecutor-4011-thread-1-processing-n:127.0.0.1:56494_q_s%2Fnl) [n:127.0.0.1:56494_q_s%2Fnl c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_54A59967B5CDEEA1-001/control-001/cores/collection1'
   [junit4]   2> 1056179 INFO  (coreLoadExecutor-4011-thread-1-processing-n:127.0.0.1:56494_q_s%2Fnl) [n:127.0.0.1:56494_q_s%2Fnl c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1056179 INFO  (coreLoadExecutor-4011-thread-1-processing-n:127.0.0.1:56494_q_s%2Fnl) [n:127.0.0.1:56494_q_s%2Fnl c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1056180 INFO  (coreLoadExecutor-4011-thread-1-processing-n:127.0.0.1:56494_q_s%2Fnl) [n:127.0.0.1:56494_q_s%2Fnl c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 1056184 INFO  (coreLoadExecutor-4011-thread-1-processing-n:127.0.0.1:56494_q_s%2Fnl) [n:127.0.0.1:56494_q_s%2Fnl c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 1056198 INFO  (coreLoadExecutor-4011-thread-1-processing-n:127.0.0.1:56494_q_s%2Fnl) [n:127.0.0.1:56494_q_s%2Fnl c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.1.0
   [junit4]   2> 1056204 INFO  (coreLoadExecutor-4011-thread-1-processing-n:127.0.0.1:56494_q_s%2Fnl) [n:127.0.0.1:56494_q_s%2Fnl c:control_collection   x:collection1] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1056209 INFO  (coreLoadExecutor-4011-thread-1-processing-n:127.0.0.1:56494_q_s%2Fnl) [n:127.0.0.1:56494_q_s%2Fnl c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1056283 INFO  (coreLoadExecutor-4011-thread-1-processing-n:127.0.0.1:56494_q_s%2Fnl) [n:127.0.0.1:56494_q_s%2Fnl c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 1056287 INFO  (coreLoadExecutor-4011-thread-1-processing-n:127.0.0.1:56494_q_s%2Fnl) [n:127.0.0.1:56494_q_s%2Fnl c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 1056288 INFO  (coreLoadExecutor-4011-thread-1-processing-n:127.0.0.1:56494_q_s%2Fnl) [n:127.0.0.1:56494_q_s%2Fnl c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 1056296 INFO  (coreLoadExecutor-4011-thread-1-processing-n:127.0.0.1:56494_q_s%2Fnl) [n:127.0.0.1:56494_q_s%2Fnl c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1056298 INFO  (coreLoadExecutor-4011-thread-1-processing-n:127.0.0.1:56494_q_s%2Fnl) [n:127.0.0.1:56494_q_s%2Fnl c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1056300 INFO  (coreLoadExecutor-4011-thread-1-processing-n:127.0.0.1:56494_q_s%2Fnl) [n:127.0.0.1:56494_q_s%2Fnl c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1056300 INFO  (coreLoadExecutor-4011-thread-1-processing-n:127.0.0.1:56494_q_s%2Fnl) [n:127.0.0.1:56494_q_s%2Fnl c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1056301 INFO  (coreLoadExecutor-4011-thread-1-processing-n:127.0.0.1:56494_q_s%2Fnl) [n:127.0.0.1:56494_q_s%2Fnl c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1
   [junit4]   2> 1056301 INFO  (coreLoadExecutor-4011-thread-1-processing-n:127.0.0.1:56494_q_s%2Fnl) [n:127.0.0.1:56494_q_s%2Fnl c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 1056302 INFO  (coreLoadExecutor-4011-thread-1-processing-n:127.0.0.1:56494_q_s%2Fnl) [n:127.0.0.1:56494_q_s%2Fnl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.HdfsDirectoryFactory
   [junit4]   2> 1056302 INFO  (coreLoadExecutor-4011-thread-1-processing-n:127.0.0.1:56494_q_s%2Fnl) [n:127.0.0.1:56494_q_s%2Fnl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:46442/solr_hdfs_home
   [junit4]   2> 1056302 INFO  (coreLoadExecutor-4011-thread-1-processing-n:127.0.0.1:56494_q_s%2Fnl) [n:127.0.0.1:56494_q_s%2Fnl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
   [junit4]   2> 1056302 INFO  (coreLoadExecutor-4011-thread-1-processing-n:127.0.0.1:56494_q_s%2Fnl) [n:127.0.0.1:56494_q_s%2Fnl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_54A59967B5CDEEA1-001/control-001/cores/collection1], dataDir=[hdfs://localhost:46442/solr_hdfs_home/control_collection/core_node1/data/]
   [junit4]   2> 1056302 INFO  (coreLoadExecutor-4011-thread-1-processing-n:127.0.0.1:56494_q_s%2Fnl) [n:127.0.0.1:56494_q_s%2Fnl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1a54c8a2
   [junit4]   2> 1056302 INFO  (coreLoadExecutor-4011-thread-1-processing-n:127.0.0.1:56494_q_s%2Fnl) [n:127.0.0.1:56494_q_s%2Fnl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:46442/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 1056316 INFO  (coreLoadExecutor-4011-thread-1-processing-n:127.0.0.1:56494_q_s%2Fnl) [n:127.0.0.1:56494_q_s%2Fnl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:46442/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 1056318 INFO  (coreLoadExecutor-4011-thread-1-processing-n:127.0.0.1:56494_q_s%2Fnl) [n:127.0.0.1:56494_q_s%2Fnl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=hdfs://localhost:46442/solr_hdfs_home/control_collection/core_node1/data/index/
   [junit4]   2> 1056325 WARN  (coreLoadExecutor-4011-thread-1-processing-n:127.0.0.1:56494_q_s%2Fnl) [n:127.0.0.1:56494_q_s%2Fnl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory 'hdfs:/localhost:46442/solr_hdfs_home/control_collection/core_node1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 1056325 INFO  (coreLoadExecutor-4011-thread-1-processing-n:127.0.0.1:56494_q_s%2Fnl) [n:127.0.0.1:56494_q_s%2Fnl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:46442/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 1056335 INFO  (coreLoadExecutor-4011-thread-1-processing-n:127.0.0.1:56494_q_s%2Fnl) [n:127.0.0.1:56494_q_s%2Fnl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:46442/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 1056341 INFO  (coreLoadExecutor-4011-thread-1-processing-n:127.0.0.1:56494_q_s%2Fnl) [n:127.0.0.1:56494_q_s%2Fnl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=34, maxMergeAtOnceExplicit=32, maxMergedSegmentMB=7.255859375, floorSegmentMB=0.7236328125, forceMergeDeletesPctAllowed=14.846088454046617, segmentsPerTier=31.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.4331034022008823
   [junit4]   2> 1056367 INFO  (IPC Server handler 1 on 46442) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:36831 is added to blk_1073741825_1001{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-5d914457-994b-4549-937c-f5c65ba8391b:NORMAL:127.0.0.1:39142|RBW], ReplicaUnderConstruction[[DISK]DS-196ac04a-ca8c-409d-9c07-cb11eef0f31b:NORMAL:127.0.0.1:36831|RBW]]} size 0
   [junit4]   2> 1056368 INFO  (IPC Server handler 7 on 46442) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:39142 is added to blk_1073741825_1001{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-196ac04a-ca8c-409d-9c07-cb11eef0f31b:NORMAL:127.0.0.1:36831|RBW], ReplicaUnderConstruction[[DISK]DS-3ccc41be-04d3-460e-987d-c074591f2af0:NORMAL:127.0.0.1:39142|FINALIZED]]} size 0
   [junit4]   2> 1056370 INFO  (coreLoadExecutor-4011-thread-1-processing-n:127.0.0.1:56494_q_s%2Fnl) [n:127.0.0.1:56494_q_s%2Fnl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(HdfsDirectory@hdfs://localhost:46442/solr_hdfs_home/control_collection/core_node1/data/index lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@6311ab85; maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 1056370 INFO  (coreLoadExecutor-4011-thread-1-processing-n:127.0.0.1:56494_q_s%2Fnl) [n:127.0.0.1:56494_q_s%2Fnl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 1056379 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:56494_q_s%2Fnl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in hdfs://localhost:46442/solr_hdfs_home/control_collection/core_node1/data/
   [junit4]   2> 1056381 INFO  (coreLoadExecutor-4011-thread-1-processing-n:127.0.0.1:56494_q_s%2Fnl) [n:127.0.0.1:56494_q_s%2Fnl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 1056381 INFO  (coreLoadExecutor-4011-thread-1-processing-n:127.0.0.1:56494_q_s%2Fnl) [n:127.0.0.1:56494_q_s%2Fnl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 1056381 INFO  (coreLoadExecutor-4011-thread-1-processing-n:127.0.0.1:56494_q_s%2Fnl) [n:127.0.0.1:56494_q_s%2Fnl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 1056381 INFO  (coreLoadExecutor-4011-thread-1-processing-n:127.0.0.1:56494_q_s%2Fnl) [n:127.0.0.1:56494_q_s%2Fnl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1056382 INFO  (coreLoadExecutor-4011-thread-1-processing-n:127.0.0.1:56494_q_s%2Fnl) [n:127.0.0.1:56494_q_s%2Fnl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1056382 INFO  (coreLoadExecutor-4011-thread-1-processing-n:127.0.0.1:56494_q_s%2Fnl) [n:127.0.0.1:56494_q_s%2Fnl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 1056382 INFO  (coreLoadExecutor-4011-thread-1-processing-n:127.0.0.1:56494_q_s%2Fnl) [n:127.0.0.1:56494_q_s%2Fnl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1056382 INFO  (coreLoadExecutor-4011-thread-1-processing-n:127.0.0.1:56494_q_s%2Fnl) [n:127.0.0.1:56494_q_s%2Fnl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1056382 INFO  (coreLoadExecutor-4011-thread-1-processing-n:127.0.0.1:56494_q_s%2Fnl) [n:127.0.0.1:56494_q_s%2Fnl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 1056382 WARN  (coreLoadExecutor-4011-thread-1-processing-n:127.0.0.1:56494_q_s%2Fnl) [n:127.0.0.1:56494_q_s%2Fnl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
   [junit4]   2> 1056383 INFO  (coreLoadExecutor-4011-thread-1-processing-n:127.0.0.1:56494_q_s%2Fnl) [n:127.0.0.1:56494_q_s%2Fnl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1056384 INFO  (coreLoadExecutor-4011-thread-1-processing-n:127.0.0.1:56494_q_s%2Fnl) [n:127.0.0.1:56494_q_s%2Fnl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1056385 INFO  (coreLoadExecutor-4011-thread-1-processing-n:127.0.0.1:56494_q_s%2Fnl) [n:127.0.0.1:56494_q_s%2Fnl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1056385 INFO  (coreLoadExecutor-4011-thread-1-processing-n:127.0.0.1:56494_q_s%2Fnl) [n:127.0.0.1:56494_q_s%2Fnl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1056387 INFO  (coreLoadExecutor-4011-thread-1-processing-n:127.0.0.1:56494_q_s%2Fnl) [n:127.0.0.1:56494_q_s%2Fnl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/mbeans,standard,/dump,/update/csv,/sql,/update/json/docs,/admin/luke,/export,/admin/segments,/get,/admin/system,/replication,/admin/properties,/config,/stream,/schema,/admin/plugins,/admin/logging,/update/json,/admin/ping,/admin/threads,/update,/admin/file
   [junit4]   2> 1056387 INFO  (coreLoadExecutor-4011-thread-1-processing-n:127.0.0.1:56494_q_s%2Fnl) [n:127.0.0.1:56494_q_s%2Fnl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 1056387 INFO  (coreLoadExecutor-4011-thread-1-processing-n:127.0.0.1:56494_q_s%2Fnl) [n:127.0.0.1:56494_q_s%2Fnl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
   [junit4]   2> 1056387 INFO  (coreLoadExecutor-4011-thread-1-processing-n:127.0.0.1:56494_q_s%2Fnl) [n:127.0.0.1:56494_q_s%2Fnl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1056388 INFO  (coreLoadExecutor-4011-thread-1-processing-n:127.0.0.1:56494_q_s%2Fnl) [n:127.0.0.1:56494_q_s%2Fnl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=2
   [junit4]   2> 1056397 INFO  (coreLoadExecutor-4011-thread-1-processing-n:127.0.0.1:56494_q_s%2Fnl) [n:127.0.0.1:56494_q_s%2Fnl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1056397 INFO  (coreLoadExecutor-4011-thread-1-processing-n:127.0.0.1:56494_q_s%2Fnl) [n:127.0.0.1:56494_q_s%2Fnl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1056398 INFO  (coreLoadExecutor-4011-thread-1-processing-n:127.0.0.1:56494_q_s%2Fnl) [n:127.0.0.1:56494_q_s%2Fnl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=5, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.3464881846070783]
   [junit4]   2> 1056404 INFO  (coreLoadExecutor-4011-thread-1-processing-n:127.0.0.1:56494_q_s%2Fnl) [n:127.0.0.1:56494_q_s%2Fnl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(HdfsDirectory@hdfs://localhost:46442/solr_hdfs_home/control_collection/core_node1/data/index lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@6311ab85; maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 1056404 INFO  (coreLoadExecutor-4011-thread-1-processing-n:127.0.0.1:56494_q_s%2Fnl) [n:127.0.0.1:56494_q_s%2Fnl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 1056404 INFO  (coreLoadExecutor-4011-thread-1-processing-n:127.0.0.1:56494_q_s%2Fnl) [n:127.0.0.1:56494_q_s%2Fnl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@ed07d90[collection1] main]
   [junit4]   2> 1056404 INFO  (coreLoadExecutor-4011-thread-1-processing-n:127.0.0.1:56494_q_s%2Fnl) [n:127.0.0.1:56494_q_s%2Fnl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/control_collection]
   [junit4]   2> 1056405 INFO  (coreLoadExecutor-4011-thread-1-processing-n:127.0.0.1:56494_q_s%2Fnl) [n:127.0.0.1:56494_q_s%2Fnl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/control_collection] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 1056405 INFO  (coreLoadExecutor-4011-thread-1-processing-n:127.0.0.1:56494_q_s%2Fnl) [n:127.0.0.1:56494_q_s%2Fnl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 1056405 INFO  (coreLoadExecutor-4011-thread-1-processing-n:127.0.0.1:56494_q_s%2Fnl) [n:127.0.0.1:56494_q_s%2Fnl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1056405 INFO  (coreLoadExecutor-4011-thread-1-processing-n:127.0.0.1:56494_q_s%2Fnl) [n:127.0.0.1:56494_q_s%2Fnl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 1056405 INFO  (coreLoadExecutor-4011-thread-1-processing-n:127.0.0.1:56494_q_s%2Fnl) [n:127.0.0.1:56494_q_s%2Fnl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1056405 INFO  (coreLoadExecutor-4011-thread-1-processing-n:127.0.0.1:56494_q_s%2Fnl) [n:127.0.0.1:56494_q_s%2Fnl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 1056406 INFO  (coreLoadExecutor-4011-thread-1-processing-n:127.0.0.1:56494_q_s%2Fnl) [n:127.0.0.1:56494_q_s%2Fnl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1056406 INFO  (coreLoadExecutor-4011-thread-1-processing-n:127.0.0.1:56494_q_s%2Fnl) [n:127.0.0.1:56494_q_s%2Fnl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 1056406 INFO  (coreLoadExecutor-4011-thread-1-processing-n:127.0.0.1:56494_q_s%2Fnl) [n:127.0.0.1:56494_q_s%2Fnl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1056407 INFO  (searcherExecutor-4012-thread-1-processing-n:127.0.0.1:56494_q_s%2Fnl x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:56494_q_s%2Fnl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@ed07d90[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1056407 INFO  (coreLoadExecutor-4011-thread-1-processing-n:127.0.0.1:56494_q_s%2Fnl) [n:127.0.0.1:56494_q_s%2Fnl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 1056407 INFO  (coreLoadExecutor-4011-thread-1-processing-n:127.0.0.1:56494_q_s%2Fnl) [n:127.0.0.1:56494_q_s%2Fnl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 1056407 INFO  (coreLoadExecutor-4011-thread-1-processing-n:127.0.0.1:56494_q_s%2Fnl) [n:127.0.0.1:56494_q_s%2Fnl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 1056407 INFO  (coreLoadExecutor-4011-thread-1-processing-n:127.0.0.1:56494_q_s%2Fnl) [n:127.0.0.1:56494_q_s%2Fnl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1533084198032637952
   [junit4]   2> 1056409 INFO  (coreLoadExecutor-4011-thread-1-processing-n:127.0.0.1:56494_q_s%2Fnl) [n:127.0.0.1:56494_q_s%2Fnl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 1.0ms to seed version buckets with highest version 1533084198032637952
   [junit4]   2> 1056409 INFO  (coreLoadExecutor-4011-thread-1-processing-n:127.0.0.1:56494_q_s%2Fnl) [n:127.0.0.1:56494_q_s%2Fnl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 1056409 INFO  (coreLoadExecutor-4011-thread-1-processing-n:127.0.0.1:56494_q_s%2Fnl) [n:127.0.0.1:56494_q_s%2Fnl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 1056409 INFO  (coreZkRegister-4005-thread-1-processing-n:127.0.0.1:56494_q_s%2Fnl x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:56494_q_s%2Fnl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:56494/q_s/nl collection:control_collection shard:shard1
   [junit4]   2> 1056410 INFO  (coreZkRegister-4005-thread-1-processing-n:127.0.0.1:56494_q_s%2Fnl x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:56494_q_s%2Fnl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 1056412 INFO  (coreZkRegister-4005-thread-1-processing-n:127.0.0.1:56494_q_s%2Fnl x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:56494_q_s%2Fnl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/95817762279849990-core_node1-n_0000000000
   [junit4]   2> 1056412 INFO  (coreZkRegister-4005-thread-1-processing-n:127.0.0.1:56494_q_s%2Fnl x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:56494_q_s%2Fnl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard=shard1 and weAreReplacement=false and leaderVoteWait=180000
   [junit4]   2> 1056413 INFO  (coreZkRegister-4005-thread-1-processing-n:127.0.0.1:56494_q_s%2Fnl x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:56494_q_s%2Fnl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1056413 INFO  (coreZkRegister-4005-thread-1-processing-n:127.0.0.1:56494_q_s%2Fnl x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:56494_q_s%2Fnl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1056413 INFO  (coreZkRegister-4005-thread-1-processing-n:127.0.0.1:56494_q_s%2Fnl x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:56494_q_s%2Fnl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:56494/q_s/nl/collection1/
   [junit4]   2> 1056413 INFO  (coreZkRegister-4005-thread-1-processing-n:127.0.0.1:56494_q_s%2Fnl x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:56494_q_s%2Fnl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 1056413 INFO  (coreZkRegister-4005-thread-1-processing-n:127.0.0.1:56494_q_s%2Fnl x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:56494_q_s%2Fnl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:56494/q_s/nl/collection1/ has no replicas
   [junit4]   2> 1056413 INFO  (OverseerStateUpdate-95817762279849990-127.0.0.1:56494_q_s%2Fnl-n_0000000000) [n:127.0.0.1:56494_q_s%2Fnl    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 1056414 INFO  (coreZkRegister-4005-thread-1-processing-n:127.0.0.1:56494_q_s%2Fnl x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:56494_q_s%2Fnl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 1056415 INFO  (coreZkRegister-4005-thread-1-processing-n:127.0.0.1:56494_q_s%2Fnl x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:56494_q_s%2Fnl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/control_collection/leaders/shard1/leader after winning as /collections/control_collection/leader_elect/shard1/election/95817762279849990-core_node1-n_0000000000
   [junit4]   2> 1056416 INFO  (coreZkRegister-4005-thread-1-processing-n:127.0.0.1:56494_q_s%2Fnl x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:56494_q_s%2Fnl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:56494/q_s/nl/collection1/ shard1
   [junit4]   2> 1056417 INFO  (OverseerStateUpdate-95817762279849990-127.0.0.1:56494_q_s%2Fnl-n_0000000000) [n:127.0.0.1:56494_q_s%2Fnl    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:56494/q_s/nl",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 1
   [junit4]   2> 1056520 INFO  (zkCallback-1665-thread-1-processing-n:127.0.0.1:56494_q_s%2Fnl) [n:127.0.0.1:56494_q_s%2Fnl    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1056567 INFO  (coreZkRegister-4005-thread-1-processing-n:127.0.0.1:56494_q_s%2Fnl x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:56494_q_s%2Fnl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:56494/q_s/nl/collection1/ and leader is http://127.0.0.1:56494/q_s/nl/collection1/
   [junit4]   2> 1056567 INFO  (coreZkRegister-4005-thread-1-processing-n:127.0.0.1:56494_q_s%2Fnl x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:56494_q_s%2Fnl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:56494/q_s/nl
   [junit4]   2> 1056567 INFO  (coreZkRegister-4005-thread-1-processing-n:127.0.0.1:56494_q_s%2Fnl x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:56494_q_s%2Fnl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1056567 INFO  (coreZkRegister-4005-thread-1-processing-n:127.0.0.1:56494_q_s%2Fnl x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:56494_q_s%2Fnl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 1056567 INFO  (coreZkRegister-4005-thread-1-processing-n:127.0.0.1:56494_q_s%2Fnl x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:56494_q_s%2Fnl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1056569 INFO  (OverseerStateUpdate-95817762279849990-127.0.0.1:56494_q_s%2Fnl-n_0000000000) [n:127.0.0.1:56494_q_s%2Fnl    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "dataDir":"hdfs://localhost:46442/solr_hdfs_home/control_collection/core_node1/data/",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:56494/q_s/nl",
   [junit4]   2>   "node_name":"127.0.0.1:56494_q_s%2Fnl",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "ulogDir":"hdfs://localhost:46442/solr_hdfs_home/control_collection/core_node1/data/tlog"} current state version: 2
   [junit4]   2> 1056569 INFO  (OverseerSt

[...truncated too long message...]

.CloudSolrClient.request(CloudSolrClient.java:898)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1595)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1616)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:987)
   [junit4]    > Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:33921/htp: KeeperErrorCode = Session expired for /overseer/collection-queue-work/qnr-
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:590)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:259)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:248)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:404)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:357)
   [junit4]    > 	... 7 moreThrowable #7: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=25232, name=collection5, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    > Caused by: java.lang.RuntimeException: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:33921/htp, http://127.0.0.1:57272/htp, http://127.0.0.1:59625/htp, http://127.0.0.1:40557/htp, http://127.0.0.1:58902/htp]
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([54A59967B5CDEEA1]:0)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:997)
   [junit4]    > Caused by: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:33921/htp, http://127.0.0.1:57272/htp, http://127.0.0.1:59625/htp, http://127.0.0.1:40557/htp, http://127.0.0.1:58902/htp]
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:384)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1192)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:962)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:898)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1595)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1616)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:987)
   [junit4]    > Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:33921/htp: KeeperErrorCode = Session expired for /overseer/collection-queue-work/qnr-
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:590)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:259)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:248)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:404)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:357)
   [junit4]    > 	... 7 more
   [junit4]   2> 2881334 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[54A59967B5CDEEA1]-worker) [    ] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_54A59967B5CDEEA1-001
   [junit4]   2> May 01, 2016 1:07:29 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene60): {other_tl1=FSTOrd50, range_facet_l_dv=PostingsFormat(name=LuceneFixedGap), _version_=FSTOrd50, multiDefault=PostingsFormat(name=MockRandom), a_t=FSTOrd50, intDefault=FSTOrd50, a_i1=FSTOrd50, id=PostingsFormat(name=LuceneFixedGap), range_facet_i_dv=FSTOrd50, text=BlockTreeOrds(blocksize=128), range_facet_l=FSTOrd50, timestamp=FSTOrd50}, docValues:{range_facet_l_dv=DocValuesFormat(name=Direct), range_facet_i_dv=DocValuesFormat(name=Asserting), timestamp=DocValuesFormat(name=Asserting)}, maxPointsInLeafNode=453, maxMBSortInHeap=5.287367129123488, sim=ClassicSimilarity, locale=pl, timezone=BET
   [junit4]   2> NOTE: Linux 3.13.0-85-generic amd64/Oracle Corporation 1.8.0_74 (64-bit)/cpus=4,threads=1,free=161921336,total=477626368
   [junit4]   2> NOTE: All tests run in this JVM: [TestFreeTextSuggestions, TestSolrQueryResponse, TestConfigSets, TestReloadAndDeleteDocs, TestSolrDynamicMBean, TestCustomSort, FacetPivotSmallTest, TestCryptoKeys, TestLFUCache, CloneFieldUpdateProcessorFactoryTest, TestUpdate, TolerantUpdateProcessorTest, TestObjectReleaseTracker, TestDynamicFieldCollectionResource, CollectionsAPIAsyncDistributedZkTest, FastVectorHighlighterTest, TestSortByMinMaxFunction, TestSolrConfigHandlerCloud, SpatialRPTFieldTypeTest, CollectionReloadTest, CloudMLTQParserTest, TestConfigSetsAPI, TestManagedSynonymFilterFactory, NumericFieldsTest, SpatialFilterTest, TestWordDelimiterFilterFactory, SolrIndexSplitterTest, ParsingFieldUpdateProcessorsTest, TestQueryTypes, TestLMJelinekMercerSimilarityFactory, SortByFunctionTest, ZkCLITest, TestExceedMaxTermLength, TestBlobHandler, TestPartialUpdateDeduplication, TestMiniSolrCloudCluster, TestQuerySenderListener, TestHashPartitioner, AsyncMigrateRouteKeyTest, SpellCheckCollatorTest, TestReRankQParserPlugin, AddSchemaFieldsUpdateProcessorFactoryTest, DistributedQueryComponentCustomSortTest, RecoveryAfterSoftCommitTest, ZkControllerTest, TestUseDocValuesAsStored, TestDFRSimilarityFactory, HttpPartitionTest, CustomCollectionTest, TestEmbeddedSolrServerConstructors, TestPivotHelperCode, TestCloudInspectUtil, TermVectorComponentTest, OpenExchangeRatesOrgProviderTest, TestDistribIDF, MBeansHandlerTest, TestSha256AuthenticationProvider, TestSchemaNameResource, WordBreakSolrSpellCheckerTest, HdfsWriteToMultipleCollectionsTest, SolrXmlInZkTest, TestSimpleTrackingShardHandler, SpellPossibilityIteratorTest, BlockJoinFacetRandomTest, TestReqParamsAPI, TestPerFieldSimilarityWithDefaultOverride, ShardRoutingCustomTest, SchemaVersionSpecificBehaviorTest, DistributedIntervalFacetingTest, TestSizeLimitedDistributedMap, DOMUtilTest, ShardRoutingTest, HdfsCollectionsAPIDistributedZkTest, SolrPluginUtilsTest, SpellingQueryConverterTest, CSVRequestHandlerTest, SoftAutoCommitTest, TestBM25SimilarityFactory, TestSolrCoreProperties, DistributedExpandComponentTest, TriLevelCompositeIdRoutingTest, SolrIndexConfigTest, TestTolerantUpdateProcessorCloud, HdfsRecoveryZkTest, MinimalSchemaTest, SharedFSAutoReplicaFailoverUtilsTest, ConvertedLegacyTest, TestAuthorizationFramework, TestDistributedGrouping, FileUtilsTest, TestQuerySenderNoQuery, SimplePostToolTest, TestHashQParserPlugin, BasicDistributedZkTest, ChaosMonkeySafeLeaderTest, BasicDistributedZk2Test, UnloadDistributedZkTest, CollectionsAPIDistributedZkTest]
   [junit4] Completed [288/592 (2!)] on J1 in 757.94s, 1 test, 1 error <<< FAILURES!

[...truncated 947 lines...]
   [junit4] JVM J1: stdout was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/temp/junit4-J1-20160501_001926_019.sysout
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] java.lang.OutOfMemoryError: GC overhead limit exceeded
   [junit4] Dumping heap to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/heapdumps/java_pid20119.hprof ...
   [junit4] Heap dump file created [610644242 bytes in 6.399 secs]
   [junit4] <<< JVM J1: EOF ----

[...truncated 23 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:747: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:684: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/common-build.xml:529: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/common-build.xml:1457: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/common-build.xml:1014: There were test failures: 592 suites (10 ignored), 2399 tests, 2 errors, 31 ignored (16 assumptions) [seed: 54A59967B5CDEEA1]

Total time: 211 minutes 8 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
No prior successful build to compare, so performing full copy of artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any