You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2016/03/23 09:34:51 UTC

[JENKINS] Lucene-Solr-NightlyTests-6.x - Build # 20 - Still Failing

Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-6.x/20/

3 tests failed.
FAILED:  org.apache.solr.cloud.DistribDocExpirationUpdateProcessorTest.test

Error Message:
Exactly one shard should have changed, instead: [shard2, shard1] nodes=([core_node3(shard2), core_node2(shard1), core_node4(shard1)]) expected:<1> but was:<2>

Stack Trace:
java.lang.AssertionError: Exactly one shard should have changed, instead: [shard2, shard1] nodes=([core_node3(shard2), core_node2(shard1), core_node4(shard1)]) expected:<1> but was:<2>
	at __randomizedtesting.SeedInfo.seed([9962B2F0FAE69E4D:11368D2A541AF3B5]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.failNotEquals(Assert.java:647)
	at org.junit.Assert.assertEquals(Assert.java:128)
	at org.junit.Assert.assertEquals(Assert.java:472)
	at org.apache.solr.cloud.DistribDocExpirationUpdateProcessorTest.test(DistribDocExpirationUpdateProcessorTest.java:118)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:996)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:971)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.handler.TestReplicationHandler.doTestStressReplication

Error Message:
[index.20160322224721130, index.20160322224716738, replication.properties, index.properties] expected:<1> but was:<2>

Stack Trace:
java.lang.AssertionError: [index.20160322224721130, index.20160322224716738, replication.properties, index.properties] expected:<1> but was:<2>
	at __randomizedtesting.SeedInfo.seed([9962B2F0FAE69E4D:42C9B236FFCEF7FE]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.failNotEquals(Assert.java:647)
	at org.junit.Assert.assertEquals(Assert.java:128)
	at org.junit.Assert.assertEquals(Assert.java:472)
	at org.apache.solr.handler.TestReplicationHandler.checkForSingleIndex(TestReplicationHandler.java:823)
	at org.apache.solr.handler.TestReplicationHandler.doTestStressReplication(TestReplicationHandler.java:790)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.test

Error Message:
Error from server at https://127.0.0.1:58984/di/j: KeeperErrorCode = Session expired for /overseer/collection-queue-work/qnr-0000000110

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:58984/di/j: KeeperErrorCode = Session expired for /overseer/collection-queue-work/qnr-0000000110
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:577)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1593)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1547)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1602)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.deleteCollectionWithDownNodes(CollectionsAPIDistributedZkTest.java:338)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test(CollectionsAPIDistributedZkTest.java:175)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:996)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:971)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 11444 lines...]
   [junit4] Suite: org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_9962B2F0FAE69E4D-001/init-core-data-001
   [junit4]   2> 221514 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[9962B2F0FAE69E4D]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (true)
   [junit4]   2> 221514 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[9962B2F0FAE69E4D]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /di/j
   [junit4]   2> 222407 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[9962B2F0FAE69E4D]-worker) [    ] o.a.h.u.NativeCodeLoader Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
   [junit4]   1> Formatting using clusterid: testClusterID
   [junit4]   2> 223392 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[9962B2F0FAE69E4D]-worker) [    ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
   [junit4]   2> 223667 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[9962B2F0FAE69E4D]-worker) [    ] o.m.log Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
   [junit4]   2> 223677 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[9962B2F0FAE69E4D]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 223770 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[9962B2F0FAE69E4D]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 223875 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[9962B2F0FAE69E4D]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/hdfs to ./temp/Jetty_localhost_57848_hdfs____.91ub98/webapp
   [junit4]   2> 224052 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[9962B2F0FAE69E4D]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 224694 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[9962B2F0FAE69E4D]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:57848
   [junit4]   2> 225822 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[9962B2F0FAE69E4D]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 225828 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[9962B2F0FAE69E4D]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 225842 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[9962B2F0FAE69E4D]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_53039_datanode____cgu59b/webapp
   [junit4]   2> 225955 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[9962B2F0FAE69E4D]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 226307 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[9962B2F0FAE69E4D]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:53039
   [junit4]   2> 226556 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[9962B2F0FAE69E4D]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 226558 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[9962B2F0FAE69E4D]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 226574 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[9962B2F0FAE69E4D]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_45591_datanode____fhd61f/webapp
   [junit4]   2> 226693 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[9962B2F0FAE69E4D]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 227053 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[9962B2F0FAE69E4D]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:45591
   [junit4]   2> 227571 INFO  (IPC Server handler 8 on 55945) [    ] BlockStateChange BLOCK* processReport: from storage DS-035ce313-4815-4120-bdbd-326331be933b node DatanodeRegistration(127.0.0.1, datanodeUuid=02f09424-4870-4452-80ef-e87764341a3e, infoPort=53039, ipcPort=52030, storageInfo=lv=-56;cid=testClusterID;nsid=92491539;c=0), blocks: 0, hasStaleStorages: true, processing time: 1 msecs
   [junit4]   2> 227571 INFO  (IPC Server handler 7 on 55945) [    ] BlockStateChange BLOCK* processReport: from storage DS-14e36be6-90fb-438a-b45b-1d787e809da2 node DatanodeRegistration(127.0.0.1, datanodeUuid=7c7e9f86-b300-4422-a720-020fbc92eaab, infoPort=45591, ipcPort=37372, storageInfo=lv=-56;cid=testClusterID;nsid=92491539;c=0), blocks: 0, hasStaleStorages: true, processing time: 4 msecs
   [junit4]   2> 227571 INFO  (IPC Server handler 8 on 55945) [    ] BlockStateChange BLOCK* processReport: from storage DS-55cf0456-721d-4204-bc68-2e75fef00779 node DatanodeRegistration(127.0.0.1, datanodeUuid=02f09424-4870-4452-80ef-e87764341a3e, infoPort=53039, ipcPort=52030, storageInfo=lv=-56;cid=testClusterID;nsid=92491539;c=0), blocks: 0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 227576 INFO  (IPC Server handler 7 on 55945) [    ] BlockStateChange BLOCK* processReport: from storage DS-d2accda1-9c60-453f-926a-b802f8f16b40 node DatanodeRegistration(127.0.0.1, datanodeUuid=7c7e9f86-b300-4422-a720-020fbc92eaab, infoPort=45591, ipcPort=37372, storageInfo=lv=-56;cid=testClusterID;nsid=92491539;c=0), blocks: 0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 228215 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 228215 INFO  (Thread-717) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 228216 INFO  (Thread-717) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 228315 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [    ] o.a.s.c.ZkTestServer start zk server on port:57219
   [junit4]   2> 228315 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 228317 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 228324 INFO  (zkCallback-279-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@634aab06 name:ZooKeeperConnection Watcher:127.0.0.1:57219 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 228326 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 228326 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 228326 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 228332 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 228333 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 228334 INFO  (zkCallback-280-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1926bf6 name:ZooKeeperConnection Watcher:127.0.0.1:57219/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 228334 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 228334 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 228334 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 228343 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 228344 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 228345 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 228346 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 228346 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 228348 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 228348 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 228350 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 228350 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 228351 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 228351 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 228352 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 228352 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 228353 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 228354 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 228355 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 228355 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 228356 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 228356 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 228357 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 228357 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 228358 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 228358 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 228359 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 228360 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 228361 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 228363 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 228364 INFO  (zkCallback-281-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@b549637 name:ZooKeeperConnection Watcher:127.0.0.1:57219/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 228364 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 228364 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 228475 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_9962B2F0FAE69E4D-001/control-001/cores/collection1
   [junit4]   2> 228479 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [    ] o.e.j.s.Server jetty-9.3.8.v20160314
   [junit4]   2> 228481 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@661e50a5{/di/j,null,AVAILABLE}
   [junit4]   2> 228482 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [    ] o.e.j.u.s.SslContextFactory x509=X509@415627b2(solrtest,h=[],w=[]) for SslContextFactory@50453f6b(file:///x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/server/etc/test/solrtest.keystore,file:///x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/server/etc/test/solrtest.keystore)
   [junit4]   2> 228487 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [    ] o.e.j.s.ServerConnector Started ServerConnector@19d9a0a6{SSL,[ssl, http/1.1]}{127.0.0.1:32799}
   [junit4]   2> 228488 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [    ] o.e.j.s.Server Started @233975ms
   [junit4]   2> 228488 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=hdfs://localhost:55945/hdfs__localhost_55945__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-6.x_solr_build_solr-core_test_J1_temp_solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_9962B2F0FAE69E4D-001_tempDir-002_control_data, hostContext=/di/j, hostPort=32799, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_9962B2F0FAE69E4D-001/control-001/cores}
   [junit4]   2> 228489 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@73d16e93
   [junit4]   2> 228489 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_9962B2F0FAE69E4D-001/control-001'
   [junit4]   2> 228489 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 228489 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 228489 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 228490 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 228497 INFO  (zkCallback-282-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@811cc28 name:ZooKeeperConnection Watcher:127.0.0.1:57219/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 228497 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 228498 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 228500 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 228500 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_9962B2F0FAE69E4D-001/control-001/solr.xml
   [junit4]   2> 228517 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_9962B2F0FAE69E4D-001/control-001/cores
   [junit4]   2> 228517 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [    ] o.a.s.c.CoreContainer New CoreContainer 155628692
   [junit4]   2> 228517 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_9962B2F0FAE69E4D-001/control-001]
   [junit4]   2> 228519 WARN  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_9962B2F0FAE69E4D-001/control-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_9962B2F0FAE69E4D-001/control-001/lib
   [junit4]   2> 228519 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 228529 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 228531 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 228531 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 228531 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:57219/solr
   [junit4]   2> 228531 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 228531 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 228532 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 228532 INFO  (zkCallback-285-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@49228d0 name:ZooKeeperConnection Watcher:127.0.0.1:57219 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 228533 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 228533 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 228535 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [n:127.0.0.1:32799_di%2Fj    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 228535 INFO  (zkCallback-286-thread-1-processing-n:127.0.0.1:32799_di%2Fj) [n:127.0.0.1:32799_di%2Fj    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@48a9c08b name:ZooKeeperConnection Watcher:127.0.0.1:57219/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 228535 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [n:127.0.0.1:32799_di%2Fj    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 228536 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [n:127.0.0.1:32799_di%2Fj    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 228538 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [n:127.0.0.1:32799_di%2Fj    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 228541 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [n:127.0.0.1:32799_di%2Fj    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 228542 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [n:127.0.0.1:32799_di%2Fj    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 228544 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [n:127.0.0.1:32799_di%2Fj    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 228547 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [n:127.0.0.1:32799_di%2Fj    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 228548 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [n:127.0.0.1:32799_di%2Fj    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 228549 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [n:127.0.0.1:32799_di%2Fj    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 228550 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [n:127.0.0.1:32799_di%2Fj    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 228550 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [n:127.0.0.1:32799_di%2Fj    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 228551 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [n:127.0.0.1:32799_di%2Fj    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (0)
   [junit4]   2> 228553 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [n:127.0.0.1:32799_di%2Fj    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 228559 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [n:127.0.0.1:32799_di%2Fj    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 228560 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [n:127.0.0.1:32799_di%2Fj    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 228560 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [n:127.0.0.1:32799_di%2Fj    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/95598495794659333-127.0.0.1:32799_di%2Fj-n_0000000000
   [junit4]   2> 228560 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [n:127.0.0.1:32799_di%2Fj    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:32799_di%2Fj
   [junit4]   2> 228560 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [n:127.0.0.1:32799_di%2Fj    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 228561 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [n:127.0.0.1:32799_di%2Fj    ] o.a.s.c.Overseer Overseer (id=95598495794659333-127.0.0.1:32799_di%2Fj-n_0000000000) starting
   [junit4]   2> 228562 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [n:127.0.0.1:32799_di%2Fj    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 228567 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [n:127.0.0.1:32799_di%2Fj    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 228572 INFO  (OverseerCollectionConfigSetProcessor-95598495794659333-127.0.0.1:32799_di%2Fj-n_0000000000) [n:127.0.0.1:32799_di%2Fj    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 228572 INFO  (OverseerStateUpdate-95598495794659333-127.0.0.1:32799_di%2Fj-n_0000000000) [n:127.0.0.1:32799_di%2Fj    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 228573 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [n:127.0.0.1:32799_di%2Fj    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:32799_di%2Fj
   [junit4]   2> 228574 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [n:127.0.0.1:32799_di%2Fj    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:32799_di%2Fj
   [junit4]   2> 228575 INFO  (zkCallback-286-thread-1-processing-n:127.0.0.1:32799_di%2Fj) [n:127.0.0.1:32799_di%2Fj    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [0])
   [junit4]   2> 228577 INFO  (OverseerStateUpdate-95598495794659333-127.0.0.1:32799_di%2Fj-n_0000000000) [n:127.0.0.1:32799_di%2Fj    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 228577 INFO  (zkCallback-286-thread-1-processing-n:127.0.0.1:32799_di%2Fj) [n:127.0.0.1:32799_di%2Fj    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (1)
   [junit4]   2> 228584 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [n:127.0.0.1:32799_di%2Fj    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 228584 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [n:127.0.0.1:32799_di%2Fj    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 228585 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [n:127.0.0.1:32799_di%2Fj    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_9962B2F0FAE69E4D-001/control-001/cores
   [junit4]   2> 228586 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [n:127.0.0.1:32799_di%2Fj    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, loadOnStartup=true, configSetProperties=configsetprops.json, collection=control_collection, coreNodeName=, dataDir=data/, shard=}
   [junit4]   2> 228586 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [n:127.0.0.1:32799_di%2Fj    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_9962B2F0FAE69E4D-001/control-001/cores/collection1
   [junit4]   2> 228586 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [n:127.0.0.1:32799_di%2Fj    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 228596 INFO  (coreLoadExecutor-901-thread-1-processing-n:127.0.0.1:32799_di%2Fj) [n:127.0.0.1:32799_di%2Fj c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 228596 INFO  (coreLoadExecutor-901-thread-1-processing-n:127.0.0.1:32799_di%2Fj) [n:127.0.0.1:32799_di%2Fj c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 228598 INFO  (coreLoadExecutor-901-thread-1-processing-n:127.0.0.1:32799_di%2Fj) [n:127.0.0.1:32799_di%2Fj c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 228610 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [n:127.0.0.1:32799_di%2Fj    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1
   [junit4]   2> 228610 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [n:127.0.0.1:32799_di%2Fj    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 228610 INFO  (OverseerStateUpdate-95598495794659333-127.0.0.1:32799_di%2Fj-n_0000000000) [n:127.0.0.1:32799_di%2Fj    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:32799/di/j",
   [junit4]   2>   "node_name":"127.0.0.1:32799_di%2Fj",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 228611 INFO  (OverseerStateUpdate-95598495794659333-127.0.0.1:32799_di%2Fj-n_0000000000) [n:127.0.0.1:32799_di%2Fj    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:32799/di/j",
   [junit4]   2>   "node_name":"127.0.0.1:32799_di%2Fj",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 228612 INFO  (OverseerStateUpdate-95598495794659333-127.0.0.1:32799_di%2Fj-n_0000000000) [n:127.0.0.1:32799_di%2Fj    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 228612 INFO  (OverseerStateUpdate-95598495794659333-127.0.0.1:32799_di%2Fj-n_0000000000) [n:127.0.0.1:32799_di%2Fj    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 228613 INFO  (zkCallback-286-thread-1-processing-n:127.0.0.1:32799_di%2Fj) [n:127.0.0.1:32799_di%2Fj    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 229598 INFO  (coreLoadExecutor-901-thread-1-processing-n:127.0.0.1:32799_di%2Fj) [n:127.0.0.1:32799_di%2Fj c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 229598 INFO  (coreLoadExecutor-901-thread-1-processing-n:127.0.0.1:32799_di%2Fj) [n:127.0.0.1:32799_di%2Fj c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 229599 INFO  (coreLoadExecutor-901-thread-1-processing-n:127.0.0.1:32799_di%2Fj) [n:127.0.0.1:32799_di%2Fj c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 229599 INFO  (coreLoadExecutor-901-thread-1-processing-n:127.0.0.1:32799_di%2Fj) [n:127.0.0.1:32799_di%2Fj c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/control_collection]
   [junit4]   2> 229599 INFO  (coreLoadExecutor-901-thread-1-processing-n:127.0.0.1:32799_di%2Fj) [n:127.0.0.1:32799_di%2Fj c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/control_collection] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 229599 INFO  (coreLoadExecutor-901-thread-1-processing-n:127.0.0.1:32799_di%2Fj) [n:127.0.0.1:32799_di%2Fj c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_9962B2F0FAE69E4D-001/control-001/cores/collection1'
   [junit4]   2> 229599 INFO  (coreLoadExecutor-901-thread-1-processing-n:127.0.0.1:32799_di%2Fj) [n:127.0.0.1:32799_di%2Fj c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 229599 INFO  (coreLoadExecutor-901-thread-1-processing-n:127.0.0.1:32799_di%2Fj) [n:127.0.0.1:32799_di%2Fj c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 229601 INFO  (coreLoadExecutor-901-thread-1-processing-n:127.0.0.1:32799_di%2Fj) [n:127.0.0.1:32799_di%2Fj c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 229617 INFO  (coreLoadExecutor-901-thread-1-processing-n:127.0.0.1:32799_di%2Fj) [n:127.0.0.1:32799_di%2Fj c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 229623 INFO  (coreLoadExecutor-901-thread-1-processing-n:127.0.0.1:32799_di%2Fj) [n:127.0.0.1:32799_di%2Fj c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.1.0
   [junit4]   2> 229635 INFO  (coreLoadExecutor-901-thread-1-processing-n:127.0.0.1:32799_di%2Fj) [n:127.0.0.1:32799_di%2Fj c:control_collection   x:collection1] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 229642 INFO  (coreLoadExecutor-901-thread-1-processing-n:127.0.0.1:32799_di%2Fj) [n:127.0.0.1:32799_di%2Fj c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 229757 INFO  (coreLoadExecutor-901-thread-1-processing-n:127.0.0.1:32799_di%2Fj) [n:127.0.0.1:32799_di%2Fj c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 229763 INFO  (coreLoadExecutor-901-thread-1-processing-n:127.0.0.1:32799_di%2Fj) [n:127.0.0.1:32799_di%2Fj c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 229764 INFO  (coreLoadExecutor-901-thread-1-processing-n:127.0.0.1:32799_di%2Fj) [n:127.0.0.1:32799_di%2Fj c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 229787 INFO  (coreLoadExecutor-901-thread-1-processing-n:127.0.0.1:32799_di%2Fj) [n:127.0.0.1:32799_di%2Fj c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 229789 INFO  (coreLoadExecutor-901-thread-1-processing-n:127.0.0.1:32799_di%2Fj) [n:127.0.0.1:32799_di%2Fj c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 229791 INFO  (coreLoadExecutor-901-thread-1-processing-n:127.0.0.1:32799_di%2Fj) [n:127.0.0.1:32799_di%2Fj c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 229791 INFO  (coreLoadExecutor-901-thread-1-processing-n:127.0.0.1:32799_di%2Fj) [n:127.0.0.1:32799_di%2Fj c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 229793 INFO  (coreLoadExecutor-901-thread-1-processing-n:127.0.0.1:32799_di%2Fj) [n:127.0.0.1:32799_di%2Fj c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1
   [junit4]   2> 229793 INFO  (coreLoadExecutor-901-thread-1-processing-n:127.0.0.1:32799_di%2Fj) [n:127.0.0.1:32799_di%2Fj c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 229793 INFO  (coreLoadExecutor-901-thread-1-processing-n:127.0.0.1:32799_di%2Fj) [n:127.0.0.1:32799_di%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.HdfsDirectoryFactory
   [junit4]   2> 229807 INFO  (coreLoadExecutor-901-thread-1-processing-n:127.0.0.1:32799_di%2Fj) [n:127.0.0.1:32799_di%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:55945/solr_hdfs_home
   [junit4]   2> 229807 INFO  (coreLoadExecutor-901-thread-1-processing-n:127.0.0.1:32799_di%2Fj) [n:127.0.0.1:32799_di%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
   [junit4]   2> 229807 INFO  (coreLoadExecutor-901-thread-1-processing-n:127.0.0.1:32799_di%2Fj) [n:127.0.0.1:32799_di%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_9962B2F0FAE69E4D-001/control-001/cores/collection1], dataDir=[hdfs://localhost:55945/solr_hdfs_home/control_collection/core_node1/data/]
   [junit4]   2> 229807 INFO  (coreLoadExecutor-901-thread-1-processing-n:127.0.0.1:32799_di%2Fj) [n:127.0.0.1:32799_di%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@37e52734
   [junit4]   2> 229807 INFO  (coreLoadExecutor-901-thread-1-processing-n:127.0.0.1:32799_di%2Fj) [n:127.0.0.1:32799_di%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:55945/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 229869 INFO  (coreLoadExecutor-901-thread-1-processing-n:127.0.0.1:32799_di%2Fj) [n:127.0.0.1:32799_di%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:55945/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 229874 INFO  (coreLoadExecutor-901-thread-1-processing-n:127.0.0.1:32799_di%2Fj) [n:127.0.0.1:32799_di%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=hdfs://localhost:55945/solr_hdfs_home/control_collection/core_node1/data/index/
   [junit4]   2> 229896 WARN  (coreLoadExecutor-901-thread-1-processing-n:127.0.0.1:32799_di%2Fj) [n:127.0.0.1:32799_di%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory 'hdfs:/localhost:55945/solr_hdfs_home/control_collection/core_node1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 229896 INFO  (coreLoadExecutor-901-thread-1-processing-n:127.0.0.1:32799_di%2Fj) [n:127.0.0.1:32799_di%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:55945/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 229919 INFO  (coreLoadExecutor-901-thread-1-processing-n:127.0.0.1:32799_di%2Fj) [n:127.0.0.1:32799_di%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:55945/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 229919 INFO  (coreLoadExecutor-901-thread-1-processing-n:127.0.0.1:32799_di%2Fj) [n:127.0.0.1:32799_di%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=22, maxMergeAtOnceExplicit=18, maxMergedSegmentMB=82.72265625, floorSegmentMB=0.236328125, forceMergeDeletesPctAllowed=8.61867506135849, segmentsPerTier=36.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.17750991538412403
   [junit4]   2> 230226 INFO  (IPC Server handler 7 on 55945) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:45842 is added to blk_1073741825_1001{blockUCState=COMMITTED, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-55cf0456-721d-4204-bc68-2e75fef00779:NORMAL:127.0.0.1:45842|RBW], ReplicaUnderConstruction[[DISK]DS-d2accda1-9c60-453f-926a-b802f8f16b40:NORMAL:127.0.0.1:56798|RBW]]} size 71
   [junit4]   2> 230227 INFO  (IPC Server handler 8 on 55945) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:56798 is added to blk_1073741825_1001 size 71
   [junit4]   2> 230633 INFO  (coreLoadExecutor-901-thread-1-processing-n:127.0.0.1:32799_di%2Fj) [n:127.0.0.1:32799_di%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(HdfsDirectory@hdfs://localhost:55945/solr_hdfs_home/control_collection/core_node1/data/index lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@56a792b3; maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 230633 INFO  (coreLoadExecutor-901-thread-1-processing-n:127.0.0.1:32799_di%2Fj) [n:127.0.0.1:32799_di%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 230647 INFO  (coreLoadExecutor-901-thread-1-processing-n:127.0.0.1:32799_di%2Fj) [n:127.0.0.1:32799_di%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 230647 INFO  (coreLoadExecutor-901-thread-1-processing-n:127.0.0.1:32799_di%2Fj) [n:127.0.0.1:32799_di%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 230647 INFO  (coreLoadExecutor-901-thread-1-processing-n:127.0.0.1:32799_di%2Fj) [n:127.0.0.1:32799_di%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 230647 INFO  (coreLoadExecutor-901-thread-1-processing-n:127.0.0.1:32799_di%2Fj) [n:127.0.0.1:32799_di%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 230647 INFO  (coreLoadExecutor-901-thread-1-processing-n:127.0.0.1:32799_di%2Fj) [n:127.0.0.1:32799_di%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 230648 INFO  (coreLoadExecutor-901-thread-1-processing-n:127.0.0.1:32799_di%2Fj) [n:127.0.0.1:32799_di%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 230648 INFO  (coreLoadExecutor-901-thread-1-processing-n:127.0.0.1:32799_di%2Fj) [n:127.0.0.1:32799_di%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 230656 INFO  (coreLoadExecutor-901-thread-1-processing-n:127.0.0.1:32799_di%2Fj) [n:127.0.0.1:32799_di%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 230656 INFO  (coreLoadExecutor-901-thread-1-processing-n:127.0.0.1:32799_di%2Fj) [n:127.0.0.1:32799_di%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 230656 WARN  (coreLoadExecutor-901-thread-1-processing-n:127.0.0.1:32799_di%2Fj) [n:127.0.0.1:32799_di%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
   [junit4]   2> 230658 INFO  (coreLoadExecutor-901-thread-1-processing-n:127.0.0.1:32799_di%2Fj) [n:127.0.0.1:32799_di%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 230659 INFO  (coreLoadExecutor-901-thread-1-processing-n:127.0.0.1:32799_di%2Fj) [n:127.0.0.1:32799_di%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 230669 INFO  (coreLoadExecutor-901-thread-1-processing-n:127.0.0.1:32799_di%2Fj) [n:127.0.0.1:32799_di%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 230670 INFO  (coreLoadExecutor-901-thread-1-processing-n:127.0.0.1:32799_di%2Fj) [n:127.0.0.1:32799_di%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 230670 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:32799_di%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in hdfs://localhost:55945/solr_hdfs_home/control_collection/core_node1/data/
   [junit4]   2> 230695 INFO  (coreLoadExecutor-901-thread-1-processing-n:127.0.0.1:32799_di%2Fj) [n:127.0.0.1:32799_di%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/mbeans,standard,/dump,/update/csv,/sql,/update/json/docs,/admin/luke,/export,/admin/segments,/get,/admin/system,/replication,/admin/properties,/config,/stream,/schema,/admin/plugins,/admin/logging,/update/json,/admin/ping,/admin/threads,/update,/admin/file
   [junit4]   2> 230695 INFO  (coreLoadExecutor-901-thread-1-processing-n:127.0.0.1:32799_di%2Fj) [n:127.0.0.1:32799_di%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 230708 INFO  (coreLoadExecutor-901-thread-1-processing-n:127.0.0.1:32799_di%2Fj) [n:127.0.0.1:32799_di%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
   [junit4]   2> 230708 INFO  (coreLoadExecutor-901-thread-1-processing-n:127.0.0.1:32799_di%2Fj) [n:127.0.0.1:32799_di%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 230708 INFO  (coreLoadExecutor-901-thread-1-processing-n:127.0.0.1:32799_di%2Fj) [n:127.0.0.1:32799_di%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=2
   [junit4]   2> 230743 INFO  (coreLoadExecutor-901-thread-1-processing-n:127.0.0.1:32799_di%2Fj) [n:127.0.0.1:32799_di%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 230743 INFO  (coreLoadExecutor-901-thread-1-processing-n:127.0.0.1:32799_di%2Fj) [n:127.0.0.1:32799_di%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 230750 INFO  (coreLoadExecutor-901-thread-1-processing-n:127.0.0.1:32799_di%2Fj) [n:127.0.0.1:32799_di%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=8, maxMergeAtOnceExplicit=6, maxMergedSegmentMB=83.59375, floorSegmentMB=0.7666015625, forceMergeDeletesPctAllowed=10.926919825893409, segmentsPerTier=12.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.5995912217994458
   [junit4]   2> 230892 INFO  (coreLoadExecutor-901-thread-1-processing-n:127.0.0.1:32799_di%2Fj) [n:127.0.0.1:32799_di%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(HdfsDirectory@hdfs://localhost:55945/solr_hdfs_home/control_collection/core_node1/data/index lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@56a792b3; maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 230892 INFO  (coreLoadExecutor-901-thread-1-processing-n:127.0.0.1:32799_di%2Fj) [n:127.0.0.1:32799_di%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 230893 INFO  (coreLoadExecutor-901-thread-1-processing-n:127.0.0.1:32799_di%2Fj) [n:127.0.0.1:32799_di%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@2c359140[collection1] main]
   [junit4]   2> 230893 INFO  (coreLoadExecutor-901-thread-1-processing-n:127.0.0.1:32799_di%2Fj) [n:127.0.0.1:32799_di%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/control_collection]
   [junit4]   2> 230894 INFO  (coreLoadExecutor-901-thread-1-processing-n:127.0.0.1:32799_di%2Fj) [n:127.0.0.1:32799_di%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/control_collection] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 230894 INFO  (coreLoadExecutor-901-thread-1-processing-n:127.0.0.1:32799_di%2Fj) [n:127.0.0.1:32799_di%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 230894 INFO  (coreLoadExecutor-901-thread-1-processing-n:127.0.0.1:32799_di%2Fj) [n:127.0.0.1:32799_di%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 230894 INFO  (coreLoadExecutor-901-thread-1-processing-n:127.0.0.1:32799_di%2Fj) [n:127.0.0.1:32799_di%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 230894 INFO  (coreLoadExecutor-901-thread-1-processing-n:127.0.0.1:32799_di%2Fj) [n:127.0.0.1:32799_di%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 230895 INFO  (coreLoadExecutor-901-thread-1-processing-n:127.0.0.1:32799_di%2Fj) [n:127.0.0.1:32799_di%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 230895 INFO  (coreLoadExecutor-901-thread-1-processing-n:127.0.0.1:32799_di%2Fj) [n:127.0.0.1:32799_di%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 230895 INFO  (coreLoadExecutor-901-thread-1-processing-n:127.0.0.1:32799_di%2Fj) [n:127.0.0.1:32799_di%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 230897 INFO  (coreLoadExecutor-901-thread-1-processing-n:127.0.0.1:32799_di%2Fj) [n:127.0.0.1:32799_di%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 230900 INFO  (searcherExecutor-902-thread-1-processing-n:127.0.0.1:32799_di%2Fj x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:32799_di%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@2c359140[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 230901 INFO  (coreLoadExecutor-901-thread-1-processing-n:127.0.0.1:32799_di%2Fj) [n:127.0.0.1:32799_di%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 230901 INFO  (coreLoadExecutor-901-thread-1-processing-n:127.0.0.1:32799_di%2Fj) [n:127.0.0.1:32799_di%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 230901 INFO  (coreLoadExecutor-901-thread-1-processing-n:127.0.0.1:32799_di%2Fj) [n:127.0.0.1:32799_di%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 230901 INFO  (coreLoadExecutor-901-thread-1-processing-n:127.0.0.1:32799_di%2Fj) [n:127.0.0.1:32799_di%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1529575935521587200
   [junit4]   2> 230903 INFO  (coreLoadExecutor-901-thread-1-processing-n:127.0.0.1:32799_di%2Fj) [n:127.0.0.1:32799_di%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 1.0ms to seed version buckets with highest version 1529575935521587200
   [junit4]   2> 230903 INFO  (coreLoadExecutor-901-thread-1-processing-n:127.0.0.1:32799_di%2Fj) [n:127.0.0.1:32799_di%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 230903 INFO  (coreLoadExecutor-901-thread-1-processing-n:127.0.0.1:32799_di%2Fj) [n:127.0.0.1:32799_di%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 230912 INFO  (coreZkRegister-895-thread-1-processing-n:127.0.0.1:32799_di%2Fj x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:32799_di%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:https://127.0.0.1:32799/di/j collection:control_collection shard:shard1
   [junit4]   2> 230913 INFO  (coreZkRegister-895-thread-1-processing-n:127.0.0.1:32799_di%2Fj x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:32799_di%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 230917 INFO  (coreZkRegister-895-thread-1-processing-n:127.0.0.1:32799_di%2Fj x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:32799_di%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/95598495794659333-core_node1-n_0000000000
   [junit4]   2> 230917 INFO  (coreZkRegister-895-thread-1-processing-n:127.0.0.1:32799_di%2Fj x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:32799_di%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard=shard1 and weAreReplacement=false and leaderVoteWait=180000
   [junit4]   2> 230919 INFO  (OverseerStateUpdate-95598495794659333-127.0.0.1:32799_di%2Fj-n_0000000000) [n:127.0.0.1:32799_di%2Fj    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 230920 INFO  (zkCallback-286-thread-1-processing-n:127.0.0.1:32799_di%2Fj) [n:127.0.0.1:32799_di%2Fj    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 230922 INFO  (coreZkRegister-895-thread-1-processing-n:127.0.0.1:32799_di%2Fj x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:32799_di%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 230922 INFO  (coreZkRegister-895-thread-1-processing-n:127.0.0.1:32799_di%2Fj x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:32799_di%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 230922 INFO  (coreZkRegister-895-thread-1-processing-n:127.0.0.1:32799_di%2Fj x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:32799_di%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:32799/di/j/collection1/
   [junit4]   2> 230922 INFO  (coreZkRegister-895-thread-1-processing-n:127.0.0.1:32799_di%2Fj x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:32799_di%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 230922 INFO  (coreZkRegister-895-thread-1-processing-n:127.0.0.1:32799_di%2Fj x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:32799_di%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy https://127.0.0.1:32799/di/j/collection1/ has no replicas
   [junit4]   2> 230923 INFO  (coreZkRegister-895-thread-1-processing-n:127.0.0.1:32799_di%2Fj x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:32799_di%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 230925 INFO  (coreZkRegister-895-thread-1-processing-n:127.0.0.1:32799_di%2Fj x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:32799_di%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/control_collection/leaders/shard1/leader after winning as /collections/control_collection/leader_elect/shard1/election/95598495794659333-core_node1-n_0000000000
   [junit4]   2> 230926 INFO  (OverseerStateUpdate-95598495794659333-127.0.0.1:32799_di%2Fj-n_0000000000) [n:127.0.0.1:32799_di%2Fj    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"https://127.0.0.1:32799/di/j",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 2
   [junit4]   2> 230928 INFO  (coreZkRegister-895-thread-1-processing-n:127.0.0.1:32799_di%2Fj x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:32799_di%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:32799/di/j/collection1/ shard1
   [junit4]   2> 231029 INFO  (zkCallback-286-thread-1-processing-n:127.0.0.1:32799_di%2Fj) [n:127.0.0.1:32799_di%2Fj    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 231030 INFO  (coreZkRegister-895-thread-1-processing-n:127.0.0.1:32799_di%2Fj x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:32799_di%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are https://127.0.0.1:32799/di/j/collection1/ and leader is https://127.0.0.1:32799/di/j/collection1/
   [junit4]   2> 231030 INFO  (coreZkRegister-895-thread-1-processing-n:127.0.0.1:32799_di%2Fj x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:32799_di%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:32799/di/j
   [junit4]   2> 231030 INFO  (coreZkRegister-895-thread-1-processing-n:127.0.0.1:32799_di%2Fj x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:32799_di%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 231030 INFO  (coreZkRegister-895-thread-1-processing-n:127.0.0.1:32799_di%2Fj x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:32799_di%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 231030 INFO  (coreZkRegister-895-thread-1-processing-n:127.0.0.1:32799_di%2Fj x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:32799_di%2Fj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 231032 INFO  (OverseerStateUpdate-95598495794659333-127.0.0.1:32799_di%2Fj-n_0000000000) [n:127.0.0.1:32799_di%2Fj    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:32799/di/j",
   [junit4]   2>   "node_name":"127.0.0.1:32799_di%2Fj",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"} current state version: 3
   [junit4]   2> 231032 INFO  (OverseerStateUpdate-95598495794659333-127.0.0.1:32799_di%2Fj-n_0000000000) [n:127.0.0.1:32799_di%2Fj    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:32799/di/j",
   [junit4]   2>   "node_name":"127.0.0.1:32799_di%2Fj",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 231116 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 231116 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 231120 INFO  (zkCallback-288-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@37ca3751 name:ZooKeeperConnection Watcher:127.0.0.1:57219/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 231120 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 231120 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 231121 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 231122 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 231122 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 231122 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with stateFormat=2
   [junit4]   2> 231122 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 231128 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 231128 INFO  (zkCallback-289-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2083022a name:ZooKeeperConnection Watcher:127.0.0.1:57219/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 231128 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 231128 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 231132 INFO  (OverseerStateUpdate-95598495794659333-127.0.0.1:32799_di%2Fj-n_0000000000) [n:127.0.0.1:32799_di%2Fj    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"create",
   [junit4]   2>   "name":"collection1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "stateFormat":"2"} current state version: 3
   [junit4]   2> 231132 INFO  (OverseerStateUpdate-95598495794659333-127.0.0.1:32799_di%2Fj-n_0000000000) [n:127.0.0.1:32799_di%2Fj    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 231133 INFO  (zkCallback-286-thread-1-processing-n:127.0.0.1:32799_di%2Fj) [n:127.0.0.1:32799_di%2Fj    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 231133 INFO  (zkCallback-288-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 231223 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_9962B2F0FAE69E4D-001/shard-1-001/cores/collection1
   [junit4]   2> 231223 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_9962B2F0FAE69E4D-001/shard-1-001
   [junit4]   2> 231225 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [    ] o.e.j.s.Server jetty-9.3.8.v20160314
   [junit4]   2> 231230 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@7acc34af{/di/j,null,AVAILABLE}
   [junit4]   2> 231231 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [    ] o.e.j.u.s.SslContextFactory x509=X509@3f6dff69(solrtest,h=[],w=[]) for SslContextFactory@3062ac63(file:///x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/server/etc/test/solrtest.keystore,file:///x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/server/etc/test/solrtest.keystore)
   [junit4]   2> 231235 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [    ] o.e.j.s.ServerConnector Started ServerConnector@46b276e4{SSL,[ssl, http/1.1]}{127.0.0.1:58984}
   [junit4]   2> 231238 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [    ] o.e.j.s.Server Started @236725ms
   [junit4]   2> 231238 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=hdfs://localhost:55945/hdfs__localhost_55945__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-6.x_solr_build_solr-core_test_J1_temp_solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_9962B2F0FAE69E4D-001_tempDir-002_jetty1, solrconfig=solrconfig.xml, hostContext=/di/j, hostPort=58984, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_9962B2F0FAE69E4D-001/shard-1-001/cores}
   [junit4]   2> 231239 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@73d16e93
   [junit4]   2> 231239 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[9962B2F0FAE69E4D]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_9962B2F0FAE69E4D-001/shard-1-001'
   [j

[...truncated too long message...]

e-027/./collection1/data/index;done=false>>]
   [junit4]   2> 2617033 INFO  (coreCloseExecutor-6461-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_9962B2F0FAE69E4D-001/solr-instance-027/./collection1/data/index
   [junit4]   2> 2617033 INFO  (coreCloseExecutor-6461-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory looking to close /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_9962B2F0FAE69E4D-001/solr-instance-027/./collection1/data [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_9962B2F0FAE69E4D-001/solr-instance-027/./collection1/data;done=false>>]
   [junit4]   2> 2617033 INFO  (coreCloseExecutor-6461-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_9962B2F0FAE69E4D-001/solr-instance-027/./collection1/data
   [junit4]   2> 2617033 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterWrite2Slave-seed#[9962B2F0FAE69E4D]) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@72165dbe{/solr,null,UNAVAILABLE}
   [junit4]   2> 2617037 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterWrite2Slave-seed#[9962B2F0FAE69E4D]) [    ] o.e.j.s.ServerConnector Stopped ServerConnector@445c342{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
   [junit4]   2> 2617037 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterWrite2Slave-seed#[9962B2F0FAE69E4D]) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=235538982
   [junit4]   2> 2617038 INFO  (coreCloseExecutor-6463-thread-1) [    x:collection1] o.a.s.c.SolrCore [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@31a1bc2f
   [junit4]   2> 2617038 INFO  (coreCloseExecutor-6463-thread-1) [    x:collection1] o.a.s.u.DirectUpdateHandler2 closing DirectUpdateHandler2{commits=5,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=5,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
   [junit4]   2> 2617038 INFO  (coreCloseExecutor-6463-thread-1) [    x:collection1] o.a.s.u.SolrCoreState Closing SolrCoreState
   [junit4]   2> 2617038 INFO  (coreCloseExecutor-6463-thread-1) [    x:collection1] o.a.s.u.DefaultSolrCoreState SolrCoreState ref count has reached 0 - closing IndexWriter
   [junit4]   2> 2617038 INFO  (coreCloseExecutor-6463-thread-1) [    x:collection1] o.a.s.u.DefaultSolrCoreState closing IndexWriter with IndexWriterCloser
   [junit4]   2> 2617038 INFO  (coreCloseExecutor-6463-thread-1) [    x:collection1] o.a.s.c.SolrCore [collection1] Closing main searcher on request.
   [junit4]   2> 2617038 INFO  (coreCloseExecutor-6463-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing MockDirectoryFactory - 2 directories currently being tracked
   [junit4]   2> 2617038 INFO  (coreCloseExecutor-6463-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory looking to close /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_9962B2F0FAE69E4D-001/solr-instance-028/./collection1/data [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_9962B2F0FAE69E4D-001/solr-instance-028/./collection1/data;done=false>>]
   [junit4]   2> 2617038 INFO  (coreCloseExecutor-6463-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_9962B2F0FAE69E4D-001/solr-instance-028/./collection1/data
   [junit4]   2> 2617039 INFO  (coreCloseExecutor-6463-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory looking to close /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_9962B2F0FAE69E4D-001/solr-instance-028/./collection1/data/index [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_9962B2F0FAE69E4D-001/solr-instance-028/./collection1/data/index;done=false>>]
   [junit4]   2> 2617039 INFO  (coreCloseExecutor-6463-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_9962B2F0FAE69E4D-001/solr-instance-028/./collection1/data/index
   [junit4]   2> 2617040 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterWrite2Slave-seed#[9962B2F0FAE69E4D]) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@6df8d2e5{/solr,null,UNAVAILABLE}
   [junit4]   2> 2617041 INFO  (SUITE-TestReplicationHandler-seed#[9962B2F0FAE69E4D]-worker) [    ] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_9962B2F0FAE69E4D-001
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene60): {name=Lucene50(blocksize=128), id=Lucene50(blocksize=128), newname=Lucene50(blocksize=128)}, docValues:{}, maxPointsInLeafNode=854, maxMBSortInHeap=4.726403366139456, sim=ClassicSimilarity, locale=nl-NL, timezone=America/Adak
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.8.0_74 (64-bit)/cpus=4,threads=1,free=220328120,total=525336576
   [junit4]   2> NOTE: All tests run in this JVM: [TestNRTOpen, DirectSolrSpellCheckerTest, ChaosMonkeySafeLeaderTest, SpellPossibilityIteratorTest, TestLMJelinekMercerSimilarityFactory, TestFieldTypeResource, OpenExchangeRatesOrgProviderTest, WordBreakSolrSpellCheckerTest, SolrIndexSplitterTest, TestSolrCoreProperties, DOMUtilTest, SoftAutoCommitTest, MinimalSchemaTest, TestCoreContainer, ZkSolrClientTest, TestFreeTextSuggestions, TestRequestForwarding, TestCodecSupport, TestAuthorizationFramework, FileUtilsTest, TestImplicitCoreProperties, TestWordDelimiterFilterFactory, TestDocumentBuilder, DocumentBuilderTest, TestSha256AuthenticationProvider, TestCustomDocTransformer, TestCustomSort, TestCloudManagedSchema, TestMissingGroups, TestBM25SimilarityFactory, HdfsCollectionsAPIDistributedZkTest, CloudMLTQParserTest, RankQueryTest, TestQueryWrapperFilter, DeleteInactiveReplicaTest, HdfsLockFactoryTest, TestScoreJoinQPNoScore, IndexBasedSpellCheckerTest, TestExceedMaxTermLength, TestDFISimilarityFactory, SchemaVersionSpecificBehaviorTest, TestCursorMarkWithoutUniqueKey, DistribDocExpirationUpdateProcessorTest, DistributedIntervalFacetingTest, TestStressUserVersions, TestFiltering, TestSQLHandler, TestConfigSetsAPIExclusivity, CheckHdfsIndexTest, TestSolrConfigHandler, SimpleCollectionCreateDeleteTest, CSVRequestHandlerTest, TestManagedResource, OverseerStatusTest, TestUpdate, TestEmbeddedSolrServerConstructors, InfoHandlerTest, SpatialFilterTest, TestFieldCollectionResource, ConvertedLegacyTest, MBeansHandlerTest, TestStressVersions, TestLRUStatsCache, TestUseDocValuesAsStored, TimeZoneUtilsTest, ZkCLITest, TestManagedStopFilterFactory, TestConfigSetProperties, UUIDUpdateProcessorFallbackTest, TestCloudPivotFacet, FastVectorHighlighterTest, RequestLoggingTest, TestReplicationHandlerBackup, TestSystemIdResolver, TestHashPartitioner, CloneFieldUpdateProcessorFactoryTest, OverriddenZkACLAndCredentialsProvidersTest, TestExpandComponent, TestSurroundQueryParser, SuggestComponentContextFilterQueryTest, MultiTermTest, QueryResultKeyTest, DistanceFunctionTest, TestDistributedGrouping, TestRawTransformer, TestJsonFacets, TestDistribDocBasedVersion, RemoteQueryErrorTest, ZkStateReaderTest, AliasIntegrationTest, TestQuerySenderListener, TermsComponentTest, ExplicitHLLTest, BasicDistributedZkTest, CollectionsAPIDistributedZkTest, OverseerTest, LeaderElectionIntegrationTest, ShardRoutingTest, RecoveryZkTest, TestReplicationHandler]
   [junit4] Completed [422/582 (3!)] on J1 in 539.53s, 13 tests, 1 failure <<< FAILURES!

[...truncated 554 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:747: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:684: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/common-build.xml:529: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/common-build.xml:1457: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/common-build.xml:1014: There were test failures: 582 suites (10 ignored), 2331 tests, 1 error, 2 failures, 32 ignored (16 assumptions) [seed: 9962B2F0FAE69E4D]

Total time: 215 minutes 18 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
No prior successful build to compare, so performing full copy of artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-NightlyTests-6.x - Build # 22 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-6.x/22/

4 tests failed.
FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=19550, name=collection3, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=19550, name=collection3, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: java.lang.RuntimeException: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:58808, http://127.0.0.1:57773, http://127.0.0.1:46696, http://127.0.0.1:53019, http://127.0.0.1:50214]
	at __randomizedtesting.SeedInfo.seed([AB4EF3E1B06AF31B]:0)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:980)
Caused by: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:58808, http://127.0.0.1:57773, http://127.0.0.1:46696, http://127.0.0.1:53019, http://127.0.0.1:50214]
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:352)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1100)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1593)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1614)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:970)
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:57773: KeeperErrorCode = Session expired for /overseer/collection-queue-work/qnr-
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:577)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	... 7 more


FAILED:  org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=5731, name=Thread-5020, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=5731, name=Thread-5020, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]
Caused by: java.lang.RuntimeException: org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:42468/collection1
	at __randomizedtesting.SeedInfo.seed([AB4EF3E1B06AF31B]:0)
	at org.apache.solr.cloud.FullSolrCloudDistribCmdsTest$1IndexThread.run(FullSolrCloudDistribCmdsTest.java:644)
Caused by: org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:42468/collection1
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:588)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:149)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:166)
	at org.apache.solr.cloud.FullSolrCloudDistribCmdsTest$1IndexThread.run(FullSolrCloudDistribCmdsTest.java:642)
Caused by: java.net.SocketTimeoutException: Read timed out
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
	at java.net.SocketInputStream.read(SocketInputStream.java:170)
	at java.net.SocketInputStream.read(SocketInputStream.java:141)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:160)
	at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:84)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:273)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
	at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
	at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)
	at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)
	at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197)
	at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)
	at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)
	at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685)
	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487)
	at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:482)
	... 5 more


FAILED:  org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.test

Error Message:
Error from server at http://127.0.0.1:39164: KeeperErrorCode = ConnectionLoss for /overseer/collection-queue-work/qnr-0000000134

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:39164: KeeperErrorCode = ConnectionLoss for /overseer/collection-queue-work/qnr-0000000134
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:577)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1593)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1614)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.deleteCollectionWithDownNodes(CollectionsAPIDistributedZkTest.java:334)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test(CollectionsAPIDistributedZkTest.java:175)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:996)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:971)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.handler.TestReplicationHandler.doTestStressReplication

Error Message:
[replication.properties, index.properties, index.20160325090005873, index.20160325090010993] expected:<1> but was:<2>

Stack Trace:
java.lang.AssertionError: [replication.properties, index.properties, index.20160325090005873, index.20160325090010993] expected:<1> but was:<2>
	at __randomizedtesting.SeedInfo.seed([AB4EF3E1B06AF31B:70E5F327B5429AA8]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.failNotEquals(Assert.java:647)
	at org.junit.Assert.assertEquals(Assert.java:128)
	at org.junit.Assert.assertEquals(Assert.java:472)
	at org.apache.solr.handler.TestReplicationHandler.checkForSingleIndex(TestReplicationHandler.java:823)
	at org.apache.solr.handler.TestReplicationHandler.doTestStressReplication(TestReplicationHandler.java:790)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 11078 lines...]
   [junit4] Suite: org.apache.solr.cloud.FullSolrCloudDistribCmdsTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_AB4EF3E1B06AF31B-001/init-core-data-001
   [junit4]   2> 122459 INFO  (SUITE-FullSolrCloudDistribCmdsTest-seed#[AB4EF3E1B06AF31B]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   2> 122462 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 122463 INFO  (Thread-4843) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 122463 INFO  (Thread-4843) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 122563 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [    ] o.a.s.c.ZkTestServer start zk server on port:49900
   [junit4]   2> 122563 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 122564 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 122566 INFO  (zkCallback-63-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7165efc8 name:ZooKeeperConnection Watcher:127.0.0.1:49900 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 122566 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 122567 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 122567 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 122573 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 122576 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 122578 INFO  (zkCallback-64-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4e461606 name:ZooKeeperConnection Watcher:127.0.0.1:49900/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 122578 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 122578 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 122578 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 122582 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 122584 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 122588 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 122589 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 122590 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 122592 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
   [junit4]   2> 122593 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 122594 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 122595 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 122600 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 122600 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 122601 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 122601 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 122603 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 122603 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 122605 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 122605 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 122606 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 122607 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 122608 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 122608 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 122610 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 122610 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 122611 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 122611 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 122708 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_AB4EF3E1B06AF31B-001/control-001/cores/collection1
   [junit4]   2> 122710 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [    ] o.e.j.s.Server jetty-9.3.8.v20160314
   [junit4]   2> 122712 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@7954b976{/,null,AVAILABLE}
   [junit4]   2> 122724 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [    ] o.e.j.s.ServerConnector Started ServerConnector@16f81b26{HTTP/1.1,[http/1.1]}{127.0.0.1:42468}
   [junit4]   2> 122724 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [    ] o.e.j.s.Server Started @125869ms
   [junit4]   2> 122724 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_AB4EF3E1B06AF31B-001/tempDir-001/control/data, hostContext=/, hostPort=42468, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_AB4EF3E1B06AF31B-001/control-001/cores}
   [junit4]   2> 122725 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@73d16e93
   [junit4]   2> 122726 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_AB4EF3E1B06AF31B-001/control-001'
   [junit4]   2> 122726 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 122726 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 122726 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 122729 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 122734 INFO  (zkCallback-65-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@41adf849 name:ZooKeeperConnection Watcher:127.0.0.1:49900/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 122734 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 122734 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 122736 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 122736 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_AB4EF3E1B06AF31B-001/control-001/solr.xml
   [junit4]   2> 122742 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_AB4EF3E1B06AF31B-001/control-001/cores
   [junit4]   2> 122742 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [    ] o.a.s.c.CoreContainer New CoreContainer 1580470030
   [junit4]   2> 122742 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_AB4EF3E1B06AF31B-001/control-001]
   [junit4]   2> 122743 WARN  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_AB4EF3E1B06AF31B-001/control-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_AB4EF3E1B06AF31B-001/control-001/lib
   [junit4]   2> 122743 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 122744 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 122745 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 122745 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 122745 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:49900/solr
   [junit4]   2> 122745 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 122745 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 122749 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 122752 INFO  (zkCallback-68-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3025b070 name:ZooKeeperConnection Watcher:127.0.0.1:49900 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 122761 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 122761 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 122764 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [n:127.0.0.1:42468_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 122765 INFO  (zkCallback-69-thread-1-processing-n:127.0.0.1:42468_) [n:127.0.0.1:42468_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6c2095bd name:ZooKeeperConnection Watcher:127.0.0.1:49900/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 122765 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [n:127.0.0.1:42468_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 122767 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [n:127.0.0.1:42468_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 122769 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [n:127.0.0.1:42468_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 122771 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [n:127.0.0.1:42468_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 122773 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [n:127.0.0.1:42468_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 122774 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [n:127.0.0.1:42468_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 122776 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [n:127.0.0.1:42468_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 122778 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [n:127.0.0.1:42468_    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 122779 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [n:127.0.0.1:42468_    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 122780 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [n:127.0.0.1:42468_    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 122781 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [n:127.0.0.1:42468_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 122784 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [n:127.0.0.1:42468_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (0)
   [junit4]   2> 122787 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [n:127.0.0.1:42468_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 122788 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [n:127.0.0.1:42468_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 122789 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [n:127.0.0.1:42468_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 122789 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [n:127.0.0.1:42468_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/95611367015841796-127.0.0.1:42468_-n_0000000000
   [junit4]   2> 122790 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [n:127.0.0.1:42468_    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:42468_
   [junit4]   2> 122790 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [n:127.0.0.1:42468_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 122791 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [n:127.0.0.1:42468_    ] o.a.s.c.Overseer Overseer (id=95611367015841796-127.0.0.1:42468_-n_0000000000) starting
   [junit4]   2> 122793 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [n:127.0.0.1:42468_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 122799 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [n:127.0.0.1:42468_    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 122800 INFO  (OverseerStateUpdate-95611367015841796-127.0.0.1:42468_-n_0000000000) [n:127.0.0.1:42468_    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 122805 INFO  (OverseerCollectionConfigSetProcessor-95611367015841796-127.0.0.1:42468_-n_0000000000) [n:127.0.0.1:42468_    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 122814 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [n:127.0.0.1:42468_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:42468_
   [junit4]   2> 122814 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [n:127.0.0.1:42468_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:42468_
   [junit4]   2> 122815 INFO  (OverseerStateUpdate-95611367015841796-127.0.0.1:42468_-n_0000000000) [n:127.0.0.1:42468_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (0)
   [junit4]   2> 122816 INFO  (zkCallback-69-thread-1-processing-n:127.0.0.1:42468_) [n:127.0.0.1:42468_    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [0])
   [junit4]   2> 122824 INFO  (zkCallback-69-thread-1-processing-n:127.0.0.1:42468_) [n:127.0.0.1:42468_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 122829 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [n:127.0.0.1:42468_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 122829 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [n:127.0.0.1:42468_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 122830 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [n:127.0.0.1:42468_    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_AB4EF3E1B06AF31B-001/control-001/cores
   [junit4]   2> 122831 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [n:127.0.0.1:42468_    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, loadOnStartup=true, configSetProperties=configsetprops.json, collection=control_collection, coreNodeName=, dataDir=data/, shard=}
   [junit4]   2> 122831 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [n:127.0.0.1:42468_    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_AB4EF3E1B06AF31B-001/control-001/cores/collection1
   [junit4]   2> 122831 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [n:127.0.0.1:42468_    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 122833 INFO  (coreLoadExecutor-136-thread-1-processing-n:127.0.0.1:42468_) [n:127.0.0.1:42468_ c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 122833 INFO  (coreLoadExecutor-136-thread-1-processing-n:127.0.0.1:42468_) [n:127.0.0.1:42468_ c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 122833 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [n:127.0.0.1:42468_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2
   [junit4]   2> 122833 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [n:127.0.0.1:42468_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 122835 INFO  (coreLoadExecutor-136-thread-1-processing-n:127.0.0.1:42468_) [n:127.0.0.1:42468_ c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 122835 INFO  (OverseerStateUpdate-95611367015841796-127.0.0.1:42468_-n_0000000000) [n:127.0.0.1:42468_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:42468",
   [junit4]   2>   "node_name":"127.0.0.1:42468_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 122835 INFO  (OverseerStateUpdate-95611367015841796-127.0.0.1:42468_-n_0000000000) [n:127.0.0.1:42468_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:42468",
   [junit4]   2>   "node_name":"127.0.0.1:42468_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 122836 INFO  (OverseerStateUpdate-95611367015841796-127.0.0.1:42468_-n_0000000000) [n:127.0.0.1:42468_    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 122836 INFO  (OverseerStateUpdate-95611367015841796-127.0.0.1:42468_-n_0000000000) [n:127.0.0.1:42468_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 122838 INFO  (zkCallback-69-thread-1-processing-n:127.0.0.1:42468_) [n:127.0.0.1:42468_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 123835 INFO  (coreLoadExecutor-136-thread-1-processing-n:127.0.0.1:42468_) [n:127.0.0.1:42468_ c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 123836 INFO  (coreLoadExecutor-136-thread-1-processing-n:127.0.0.1:42468_) [n:127.0.0.1:42468_ c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 123836 INFO  (coreLoadExecutor-136-thread-1-processing-n:127.0.0.1:42468_) [n:127.0.0.1:42468_ c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 123836 INFO  (coreLoadExecutor-136-thread-1-processing-n:127.0.0.1:42468_) [n:127.0.0.1:42468_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/control_collection]
   [junit4]   2> 123837 INFO  (coreLoadExecutor-136-thread-1-processing-n:127.0.0.1:42468_) [n:127.0.0.1:42468_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/control_collection] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 123837 INFO  (coreLoadExecutor-136-thread-1-processing-n:127.0.0.1:42468_) [n:127.0.0.1:42468_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_AB4EF3E1B06AF31B-001/control-001/cores/collection1'
   [junit4]   2> 123837 INFO  (coreLoadExecutor-136-thread-1-processing-n:127.0.0.1:42468_) [n:127.0.0.1:42468_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 123837 INFO  (coreLoadExecutor-136-thread-1-processing-n:127.0.0.1:42468_) [n:127.0.0.1:42468_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 123839 INFO  (coreLoadExecutor-136-thread-1-processing-n:127.0.0.1:42468_) [n:127.0.0.1:42468_ c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 123846 INFO  (coreLoadExecutor-136-thread-1-processing-n:127.0.0.1:42468_) [n:127.0.0.1:42468_ c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 123850 INFO  (coreLoadExecutor-136-thread-1-processing-n:127.0.0.1:42468_) [n:127.0.0.1:42468_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.1.0
   [junit4]   2> 123867 INFO  (coreLoadExecutor-136-thread-1-processing-n:127.0.0.1:42468_) [n:127.0.0.1:42468_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 123881 INFO  (coreLoadExecutor-136-thread-1-processing-n:127.0.0.1:42468_) [n:127.0.0.1:42468_ c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 123966 INFO  (coreLoadExecutor-136-thread-1-processing-n:127.0.0.1:42468_) [n:127.0.0.1:42468_ c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 123967 INFO  (coreLoadExecutor-136-thread-1-processing-n:127.0.0.1:42468_) [n:127.0.0.1:42468_ c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 123968 INFO  (coreLoadExecutor-136-thread-1-processing-n:127.0.0.1:42468_) [n:127.0.0.1:42468_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 123971 INFO  (coreLoadExecutor-136-thread-1-processing-n:127.0.0.1:42468_) [n:127.0.0.1:42468_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 124002 INFO  (coreLoadExecutor-136-thread-1-processing-n:127.0.0.1:42468_) [n:127.0.0.1:42468_ c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2
   [junit4]   2> 124002 INFO  (coreLoadExecutor-136-thread-1-processing-n:127.0.0.1:42468_) [n:127.0.0.1:42468_ c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 124002 INFO  (coreLoadExecutor-136-thread-1-processing-n:127.0.0.1:42468_) [n:127.0.0.1:42468_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 124004 INFO  (coreLoadExecutor-136-thread-1-processing-n:127.0.0.1:42468_) [n:127.0.0.1:42468_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_AB4EF3E1B06AF31B-001/control-001/cores/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_AB4EF3E1B06AF31B-001/control-001/cores/collection1/data/]
   [junit4]   2> 124004 INFO  (coreLoadExecutor-136-thread-1-processing-n:127.0.0.1:42468_) [n:127.0.0.1:42468_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@3a1bdee1
   [junit4]   2> 124029 INFO  (coreLoadExecutor-136-thread-1-processing-n:127.0.0.1:42468_) [n:127.0.0.1:42468_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_AB4EF3E1B06AF31B-001/control-001/cores/collection1/data
   [junit4]   2> 124031 INFO  (coreLoadExecutor-136-thread-1-processing-n:127.0.0.1:42468_) [n:127.0.0.1:42468_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_AB4EF3E1B06AF31B-001/control-001/cores/collection1/data/index/
   [junit4]   2> 124031 WARN  (coreLoadExecutor-136-thread-1-processing-n:127.0.0.1:42468_) [n:127.0.0.1:42468_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_AB4EF3E1B06AF31B-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 124032 INFO  (coreLoadExecutor-136-thread-1-processing-n:127.0.0.1:42468_) [n:127.0.0.1:42468_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_AB4EF3E1B06AF31B-001/control-001/cores/collection1/data/index
   [junit4]   2> 124035 INFO  (coreLoadExecutor-136-thread-1-processing-n:127.0.0.1:42468_) [n:127.0.0.1:42468_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=16, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.6776218721792453]
   [junit4]   2> 124039 INFO  (coreLoadExecutor-136-thread-1-processing-n:127.0.0.1:42468_) [n:127.0.0.1:42468_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@296dff7c lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@54b4f7c0),segFN=segments_1,generation=1}
   [junit4]   2> 124039 INFO  (coreLoadExecutor-136-thread-1-processing-n:127.0.0.1:42468_) [n:127.0.0.1:42468_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 124040 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:42468_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_AB4EF3E1B06AF31B-001/control-001/cores/collection1/data/
   [junit4]   2> 124041 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:42468_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_AB4EF3E1B06AF31B-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 124047 INFO  (coreLoadExecutor-136-thread-1-processing-n:127.0.0.1:42468_) [n:127.0.0.1:42468_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 124047 INFO  (coreLoadExecutor-136-thread-1-processing-n:127.0.0.1:42468_) [n:127.0.0.1:42468_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 124047 INFO  (coreLoadExecutor-136-thread-1-processing-n:127.0.0.1:42468_) [n:127.0.0.1:42468_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 124047 INFO  (coreLoadExecutor-136-thread-1-processing-n:127.0.0.1:42468_) [n:127.0.0.1:42468_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 124048 INFO  (coreLoadExecutor-136-thread-1-processing-n:127.0.0.1:42468_) [n:127.0.0.1:42468_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 124048 INFO  (coreLoadExecutor-136-thread-1-processing-n:127.0.0.1:42468_) [n:127.0.0.1:42468_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 124048 INFO  (coreLoadExecutor-136-thread-1-processing-n:127.0.0.1:42468_) [n:127.0.0.1:42468_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 124048 INFO  (coreLoadExecutor-136-thread-1-processing-n:127.0.0.1:42468_) [n:127.0.0.1:42468_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 124048 INFO  (coreLoadExecutor-136-thread-1-processing-n:127.0.0.1:42468_) [n:127.0.0.1:42468_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 124048 WARN  (coreLoadExecutor-136-thread-1-processing-n:127.0.0.1:42468_) [n:127.0.0.1:42468_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
   [junit4]   2> 124051 INFO  (coreLoadExecutor-136-thread-1-processing-n:127.0.0.1:42468_) [n:127.0.0.1:42468_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 124052 INFO  (coreLoadExecutor-136-thread-1-processing-n:127.0.0.1:42468_) [n:127.0.0.1:42468_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 124053 INFO  (coreLoadExecutor-136-thread-1-processing-n:127.0.0.1:42468_) [n:127.0.0.1:42468_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 124054 INFO  (coreLoadExecutor-136-thread-1-processing-n:127.0.0.1:42468_) [n:127.0.0.1:42468_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 124058 INFO  (coreLoadExecutor-136-thread-1-processing-n:127.0.0.1:42468_) [n:127.0.0.1:42468_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/mbeans,standard,/dump,/update/csv,/sql,/update/json/docs,/admin/luke,/export,/admin/segments,/get,/admin/system,/replication,/admin/properties,/config,/stream,/schema,/admin/plugins,/admin/logging,/update/json,/admin/ping,/admin/threads,/update,/admin/file
   [junit4]   2> 124058 INFO  (coreLoadExecutor-136-thread-1-processing-n:127.0.0.1:42468_) [n:127.0.0.1:42468_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 124059 INFO  (coreLoadExecutor-136-thread-1-processing-n:127.0.0.1:42468_) [n:127.0.0.1:42468_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 124059 INFO  (coreLoadExecutor-136-thread-1-processing-n:127.0.0.1:42468_) [n:127.0.0.1:42468_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 124060 INFO  (coreLoadExecutor-136-thread-1-processing-n:127.0.0.1:42468_) [n:127.0.0.1:42468_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 124060 INFO  (coreLoadExecutor-136-thread-1-processing-n:127.0.0.1:42468_) [n:127.0.0.1:42468_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 124061 INFO  (coreLoadExecutor-136-thread-1-processing-n:127.0.0.1:42468_) [n:127.0.0.1:42468_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@3dbc12ad
   [junit4]   2> 124064 INFO  (coreLoadExecutor-136-thread-1-processing-n:127.0.0.1:42468_) [n:127.0.0.1:42468_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@296dff7c lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@54b4f7c0),segFN=segments_1,generation=1}
   [junit4]   2> 124064 INFO  (coreLoadExecutor-136-thread-1-processing-n:127.0.0.1:42468_) [n:127.0.0.1:42468_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 124065 INFO  (coreLoadExecutor-136-thread-1-processing-n:127.0.0.1:42468_) [n:127.0.0.1:42468_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@769a3d44[collection1] main]
   [junit4]   2> 124065 INFO  (coreLoadExecutor-136-thread-1-processing-n:127.0.0.1:42468_) [n:127.0.0.1:42468_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/control_collection]
   [junit4]   2> 124065 INFO  (coreLoadExecutor-136-thread-1-processing-n:127.0.0.1:42468_) [n:127.0.0.1:42468_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/control_collection] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 124066 INFO  (coreLoadExecutor-136-thread-1-processing-n:127.0.0.1:42468_) [n:127.0.0.1:42468_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 124066 INFO  (coreLoadExecutor-136-thread-1-processing-n:127.0.0.1:42468_) [n:127.0.0.1:42468_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 124066 INFO  (coreLoadExecutor-136-thread-1-processing-n:127.0.0.1:42468_) [n:127.0.0.1:42468_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 124066 INFO  (coreLoadExecutor-136-thread-1-processing-n:127.0.0.1:42468_) [n:127.0.0.1:42468_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 124066 INFO  (coreLoadExecutor-136-thread-1-processing-n:127.0.0.1:42468_) [n:127.0.0.1:42468_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 124066 INFO  (coreLoadExecutor-136-thread-1-processing-n:127.0.0.1:42468_) [n:127.0.0.1:42468_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 124066 INFO  (coreLoadExecutor-136-thread-1-processing-n:127.0.0.1:42468_) [n:127.0.0.1:42468_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 124067 INFO  (coreLoadExecutor-136-thread-1-processing-n:127.0.0.1:42468_) [n:127.0.0.1:42468_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 124067 INFO  (coreLoadExecutor-136-thread-1-processing-n:127.0.0.1:42468_) [n:127.0.0.1:42468_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 124067 INFO  (coreLoadExecutor-136-thread-1-processing-n:127.0.0.1:42468_) [n:127.0.0.1:42468_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 124067 INFO  (coreLoadExecutor-136-thread-1-processing-n:127.0.0.1:42468_) [n:127.0.0.1:42468_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 124067 INFO  (coreLoadExecutor-136-thread-1-processing-n:127.0.0.1:42468_) [n:127.0.0.1:42468_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1529781873933287424
   [junit4]   2> 124069 INFO  (coreLoadExecutor-136-thread-1-processing-n:127.0.0.1:42468_) [n:127.0.0.1:42468_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 1.0ms to seed version buckets with highest version 1529781873933287424
   [junit4]   2> 124069 INFO  (coreLoadExecutor-136-thread-1-processing-n:127.0.0.1:42468_) [n:127.0.0.1:42468_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 124070 INFO  (coreLoadExecutor-136-thread-1-processing-n:127.0.0.1:42468_) [n:127.0.0.1:42468_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 124070 INFO  (searcherExecutor-137-thread-1-processing-n:127.0.0.1:42468_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:42468_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@769a3d44[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 124073 INFO  (coreZkRegister-130-thread-1-processing-n:127.0.0.1:42468_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:42468_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:42468 collection:control_collection shard:shard1
   [junit4]   2> 124075 INFO  (coreZkRegister-130-thread-1-processing-n:127.0.0.1:42468_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:42468_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 124080 INFO  (coreZkRegister-130-thread-1-processing-n:127.0.0.1:42468_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:42468_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/95611367015841796-core_node1-n_0000000000
   [junit4]   2> 124080 INFO  (coreZkRegister-130-thread-1-processing-n:127.0.0.1:42468_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:42468_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard=shard1 and weAreReplacement=false and leaderVoteWait=10000
   [junit4]   2> 124084 INFO  (coreZkRegister-130-thread-1-processing-n:127.0.0.1:42468_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:42468_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 124084 INFO  (coreZkRegister-130-thread-1-processing-n:127.0.0.1:42468_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:42468_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 124084 INFO  (OverseerStateUpdate-95611367015841796-127.0.0.1:42468_-n_0000000000) [n:127.0.0.1:42468_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 124084 INFO  (coreZkRegister-130-thread-1-processing-n:127.0.0.1:42468_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:42468_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:42468/collection1/
   [junit4]   2> 124084 INFO  (coreZkRegister-130-thread-1-processing-n:127.0.0.1:42468_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:42468_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 124084 INFO  (coreZkRegister-130-thread-1-processing-n:127.0.0.1:42468_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:42468_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:42468/collection1/ has no replicas
   [junit4]   2> 124086 INFO  (coreZkRegister-130-thread-1-processing-n:127.0.0.1:42468_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:42468_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 124091 INFO  (coreZkRegister-130-thread-1-processing-n:127.0.0.1:42468_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:42468_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/control_collection/leaders/shard1/leader after winning as /collections/control_collection/leader_elect/shard1/election/95611367015841796-core_node1-n_0000000000
   [junit4]   2> 124098 INFO  (coreZkRegister-130-thread-1-processing-n:127.0.0.1:42468_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:42468_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:42468/collection1/ shard1
   [junit4]   2> 124098 INFO  (OverseerStateUpdate-95611367015841796-127.0.0.1:42468_-n_0000000000) [n:127.0.0.1:42468_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:42468",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 1
   [junit4]   2> 124201 INFO  (zkCallback-69-thread-1-processing-n:127.0.0.1:42468_) [n:127.0.0.1:42468_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 124250 INFO  (coreZkRegister-130-thread-1-processing-n:127.0.0.1:42468_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:42468_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:42468/collection1/ and leader is http://127.0.0.1:42468/collection1/
   [junit4]   2> 124250 INFO  (coreZkRegister-130-thread-1-processing-n:127.0.0.1:42468_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:42468_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:42468
   [junit4]   2> 124250 INFO  (coreZkRegister-130-thread-1-processing-n:127.0.0.1:42468_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:42468_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 124250 INFO  (coreZkRegister-130-thread-1-processing-n:127.0.0.1:42468_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:42468_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 124250 INFO  (coreZkRegister-130-thread-1-processing-n:127.0.0.1:42468_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:42468_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 124253 INFO  (OverseerStateUpdate-95611367015841796-127.0.0.1:42468_-n_0000000000) [n:127.0.0.1:42468_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:42468",
   [junit4]   2>   "node_name":"127.0.0.1:42468_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"} current state version: 2
   [junit4]   2> 124254 INFO  (OverseerStateUpdate-95611367015841796-127.0.0.1:42468_-n_0000000000) [n:127.0.0.1:42468_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:42468",
   [junit4]   2>   "node_name":"127.0.0.1:42468_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 124335 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 124337 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 124345 INFO  (zkCallback-71-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1661d333 name:ZooKeeperConnection Watcher:127.0.0.1:49900/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 124345 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 124346 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 124346 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 124347 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 124348 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 124356 INFO  (zkCallback-69-thread-1-processing-n:127.0.0.1:42468_) [n:127.0.0.1:42468_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 124356 INFO  (zkCallback-71-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 124435 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_AB4EF3E1B06AF31B-001/shard-1-001/cores/collection1
   [junit4]   2> 124436 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_AB4EF3E1B06AF31B-001/shard-1-001
   [junit4]   2> 124437 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [    ] o.e.j.s.Server jetty-9.3.8.v20160314
   [junit4]   2> 124438 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@53fea7c9{/,null,AVAILABLE}
   [junit4]   2> 124439 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [    ] o.e.j.s.ServerConnector Started ServerConnector@3deed4e1{HTTP/1.1,[http/1.1]}{127.0.0.1:56275}
   [junit4]   2> 124439 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [    ] o.e.j.s.Server Started @127584ms
   [junit4]   2> 124439 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_AB4EF3E1B06AF31B-001/tempDir-001/jetty1, solrconfig=solrconfig.xml, hostContext=/, hostPort=56275, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_AB4EF3E1B06AF31B-001/shard-1-001/cores}
   [junit4]   2> 124440 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@73d16e93
   [junit4]   2> 124440 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_AB4EF3E1B06AF31B-001/shard-1-001'
   [junit4]   2> 124440 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 124440 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 124440 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 124441 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 124449 INFO  (zkCallback-72-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@53e0d6b name:ZooKeeperConnection Watcher:127.0.0.1:49900/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 124449 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 124449 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 124452 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 124453 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_AB4EF3E1B06AF31B-001/shard-1-001/solr.xml
   [junit4]   2> 124462 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_AB4EF3E1B06AF31B-001/shard-1-001/cores
   [junit4]   2> 124462 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [    ] o.a.s.c.CoreContainer New CoreContainer 82512329
   [junit4]   2> 124462 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_AB4EF3E1B06AF31B-001/shard-1-001]
   [junit4]   2> 124462 WARN  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_AB4EF3E1B06AF31B-001/shard-1-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_AB4EF3E1B06AF31B-001/shard-1-001/lib
   [junit4]   2> 124462 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 124464 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 124464 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 124464 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 124464 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:49900/solr
   [junit4]   2> 124464 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 124465 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 124473 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 124477 INFO  (zkCallback-75-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3549b35b name:ZooKeeperConnection Watcher:127.0.0.1:49900 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 124478 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 124478 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 124486 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [n:127.0.0.1:56275_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 124496 INFO  (zkCallback-76-thread-1-processing-n:127.0.0.1:56275_) [n:127.0.0.1:56275_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3b26bd92 name:ZooKeeperConnection Watcher:127.0.0.1:49900/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 124496 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [n:127.0.0.1:56275_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 124506 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [n:127.0.0.1:56275_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 124507 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [n:127.0.0.1:56275_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 124509 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [n:127.0.0.1:56275_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 124509 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [n:127.0.0.1:56275_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/95611367015841800-127.0.0.1:56275_-n_0000000001
   [junit4]   2> 124510 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [n:127.0.0.1:56275_    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/95611367015841796-127.0.0.1:42468_-n_0000000000 to know if I could be the leader
   [junit4]   2> 124510 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [n:127.0.0.1:56275_    ] o.a.s.c.ZkController Publish node=127.0.0.1:56275_ as DOWN
   [junit4]   2> 124512 INFO  (OverseerStateUpdate-95611367015841796-127.0.0.1:42468_-n_0000000000) [n:127.0.0.1:42468_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"downnode",
   [junit4]   2>   "node_name":"127.0.0.1:56275_"} current state version: 3
   [junit4]   2> 124512 INFO  (OverseerStateUpdate-95611367015841796-127.0.0.1:42468_-n_0000000000) [n:127.0.0.1:42468_    ] o.a.s.c.o.NodeMutator DownNode state invoked for node: 127.0.0.1:56275_
   [junit4]   2> 124614 INFO  (zkCallback-76-thread-1-processing-n:127.0.0.1:56275_) [n:127.0.0.1:56275_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 124614 INFO  (zkCallback-71-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 124614 INFO  (zkCallback-69-thread-1-processing-n:127.0.0.1:42468_) [n:127.0.0.1:42468_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 125512 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [n:127.0.0.1:56275_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:56275_
   [junit4]   2> 125513 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [n:127.0.0.1:56275_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:56275_
   [junit4]   2> 125514 INFO  (zkCallback-76-thread-1-processing-n:127.0.0.1:56275_) [n:127.0.0.1:56275_    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 125514 INFO  (zkCallback-71-thread-1) [    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 125514 INFO  (zkCallback-69-thread-1-processing-n:127.0.0.1:42468_) [n:127.0.0.1:42468_    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 125515 INFO  (zkCallback-71-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 125515 INFO  (zkCallback-76-thread-1-processing-n:127.0.0.1:56275_) [n:127.0.0.1:56275_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 125519 INFO  (zkCallback-69-thread-1-processing-n:127.0.0.1:42468_) [n:127.0.0.1:42468_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 125523 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [n:127.0.0.1:56275_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 125523 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [n:127.0.0.1:56275_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 125524 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [n:127.0.0.1:56275_    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_AB4EF3E1B06AF31B-001/shard-1-001/cores
   [junit4]   2> 125525 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [n:127.0.0.1:56275_    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, loadOnStartup=true, configSetProperties=configsetprops.json, collection=collection1, coreNodeName=, dataDir=data/, shard=}
   [junit4]   2> 125525 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [n:127.0.0.1:56275_    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_AB4EF3E1B06AF31B-001/shard-1-001/cores/collection1
   [junit4]   2> 125525 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [n:127.0.0.1:56275_    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 125531 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[AB4EF3E1B06AF31B]) [n:127.0.0.1:56275_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-cor

[...truncated too long message...]

2> 	at org.apache.solr.common.cloud.SolrZkClient$3.process(SolrZkClient.java:266)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
   [junit4]   2> 8693644 ERROR (zkCallback-4073-thread-7-processing-n:127.0.0.1:38140_-EventThread) [n:127.0.0.1:38140_    ] o.a.z.ClientCnxn Error while calling watcher 
   [junit4]   2> java.util.concurrent.RejectedExecutionException: Task org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$7/125936944@6887dc3f rejected from org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor@7566214e[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 521]
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2047)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:823)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1369)
   [junit4]   2> 	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.execute(ExecutorUtil.java:214)
   [junit4]   2> 	at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$3.process(SolrZkClient.java:266)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
   [junit4]   2> 8693644 ERROR (zkCallback-4073-thread-7-processing-n:127.0.0.1:38140_-EventThread) [n:127.0.0.1:38140_    ] o.a.z.ClientCnxn Error while calling watcher 
   [junit4]   2> java.util.concurrent.RejectedExecutionException: Task org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$7/125936944@15f4843a rejected from org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor@7566214e[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 521]
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2047)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:823)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1369)
   [junit4]   2> 	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.execute(ExecutorUtil.java:214)
   [junit4]   2> 	at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$3.process(SolrZkClient.java:266)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
   [junit4]   2> NOTE: test params are: codec=Lucene60, sim=ClassicSimilarity, locale=th-TH-u-nu-thai-x-lvariant-TH, timezone=Asia/Novokuznetsk
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.8.0_74 (64-bit)/cpus=4,threads=2,free=25376264,total=423100416
   [junit4]   2> NOTE: All tests run in this JVM: [TlogReplayBufferedWhileIndexingTest, CollectionTooManyReplicasTest, BlockCacheTest, BlockJoinFacetDistribTest, DistributedFacetPivotSmallTest, CustomCollectionTest, DeleteReplicaTest, SegmentsInfoRequestHandlerTest, BitVectorTest, DistributedFacetPivotLargeTest, XsltUpdateRequestHandlerTest, SOLR749Test, PolyFieldTest, TestReplicationHandler, TestComplexPhraseQParserPlugin, AnalysisErrorHandlingTest, BadCopyFieldTest, TestBinaryField, TestClassicSimilarityFactory, TestMiniSolrCloudClusterSSL, TestTestInjection, TestLuceneMatchVersion, TestReplicaProperties, TestRangeQuery, RestartWhileUpdatingTest, SynonymTokenizerTest, TestShortCircuitedRequests, TestMaxScoreQueryParser, MergeStrategyTest, TestFoldingMultitermQuery, OutputWriterTest, OverseerRolesTest, TestConfig, BasicDistributedZkTest, TestSortingResponseWriter, DefaultValueUpdateProcessorTest, TestFastOutputStream, TestConfigSetImmutable, GraphQueryTest, TestRestManager, RAMDirectoryFactoryTest, TestLazyCores, TestDocSet, TestSerializedLuceneMatchVersion, DistributedSpellCheckComponentTest, HdfsSyncSliceTest, CacheHeaderTest, ResourceLoaderTest, CachingDirectoryFactoryTest, AnalysisAfterCoreReloadTest, TestCryptoKeys, TestTrackingShardHandlerFactory, IgnoreCommitOptimizeUpdateProcessorFactoryTest, RecoveryAfterSoftCommitTest, SolrRequestParserTest, TestStandardQParsers, DocumentAnalysisRequestHandlerTest, JsonLoaderTest, EchoParamsTest, TestCloudInspectUtil, TestFilteredDocIdSet, ConnectionReuseTest, PKIAuthenticationIntegrationTest, TestRandomDVFaceting, SharedFSAutoReplicaFailoverUtilsTest, TestFileDictionaryLookup, TestRTGBase, DistributedQueueTest, ZkNodePropsTest, DateFieldTest, TestPerFieldSimilarityClassic, ReturnFieldsTest, CurrencyFieldOpenExchangeTest, TestComponentsName, ExternalFileFieldSortTest, TestReloadDeadlock, TestLMDirichletSimilarityFactory, DocValuesMissingTest, AsyncCallRequestStatusResponseTest, TestFieldTypeCollectionResource, TestCrossCoreJoin, TestOrdValues, TestPhraseSuggestions, ResponseLogComponentTest, PrimUtilsTest, SpatialHeatmapFacetsTest, HdfsBasicDistributedZk2Test, TestSolrXml, TestShardHandlerFactory, PeerSyncTest, LeaderElectionIntegrationTest, IndexSchemaRuntimeFieldTest, TestSolrConfigHandlerConcurrent, OverseerTest, LeaderElectionTest, ZkCLITest, ShardRoutingCustomTest, AutoCommitTest, TestDistributedGrouping, TestFaceting, TestRecovery, TestHashPartitioner, TermVectorComponentDistributedTest, ZkControllerTest, TestRealTimeGet, TestJoin, TestStressVersions, TestSolr4Spatial, SolrCoreTest, SolrCmdDistributorTest, QueryElevationComponentTest, BadIndexSchemaTest, ConvertedLegacyTest, TestSort, TestFiltering, TestFunctionQuery, TestBadConfig, TestIndexSearcher, HighlighterTest, SoftAutoCommitTest, DistributedQueryElevationComponentTest, SimplePostToolTest, SignatureUpdateProcessorFactoryTest, TestExtendedDismaxParser, SuggesterFSTTest, CoreAdminHandlerTest, DocValuesTest, TestStressLucene, TestTrie, SpatialFilterTest, SuggesterWFSTTest, TestCSVLoader, NoCacheHeaderTest, SchemaVersionSpecificBehaviorTest, TestPseudoReturnFields, TestUpdate, FieldMutatingUpdateProcessorTest, TestAtomicUpdateErrorCases, TestWordDelimiterFilterFactory, QueryEqualityTest, DirectUpdateHandlerOptimizeTest, SortByFunctionTest, TestRemoteStreaming, IndexBasedSpellCheckerTest, LukeRequestHandlerTest, DisMaxRequestHandlerTest, TestQueryUtils, StandardRequestHandlerTest, TestWriterPerf, TestReversedWildcardFilterFactory, TestOmitPositions, FileBasedSpellCheckerTest, RequestHandlersTest, DocumentBuilderTest, TestValueSourceCache, PathHierarchyTokenizerFactoryTest, TermVectorComponentTest, TestIndexingPerformance, RequiredFieldsTest, FieldAnalysisRequestHandlerTest, TestArbitraryIndexDir, RegexBoostProcessorTest, SolrPluginUtilsTest, IndexSchemaTest, TestJmxIntegration, JSONWriterTest, TestCSVResponseWriter, MBeansHandlerTest, TestAnalyzedSuggestions, UniqFieldsUpdateProcessorFactoryTest, TestPartialUpdateDeduplication, TestPHPSerializedResponseWriter, PingRequestHandlerTest, TestBinaryResponseWriter, HighlighterConfigTest, TestQuerySenderListener, AlternateDirectoryTest, TestSolrIndexConfig, CopyFieldTest, BadComponentTest, TestStressRecovery, TestMergePolicyConfig, TestElisionMultitermQuery, SpellPossibilityIteratorTest, TestCodecSupport, TestPerFieldSimilarity, TestLMJelinekMercerSimilarityFactory, TestBM25SimilarityFactory, TimeZoneUtilsTest, URLClassifyProcessorTest, DateMathParserTest, TestSuggestSpellingConverter, TestUtils, FileUtilsTest, CursorPagingTest, SolrTestCaseJ4Test, TestDistributedMissingSort, TestSimpleTrackingShardHandler, CdcrReplicationDistributedZkTest, HdfsCollectionsAPIDistributedZkTest]
   [junit4] Completed [582/582 (4!)] on J0 in 4723.81s, 1 test, 1 error <<< FAILURES!

[...truncated 1 lines...]
   [junit4] JVM J0: stdout was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/temp/junit4-J0-20160325_134555_607.sysout
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] java.lang.OutOfMemoryError: Java heap space
   [junit4] Dumping heap to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/heapdumps/java_pid32126.hprof ...
   [junit4] Heap dump file created [611464063 bytes in 4.618 secs]
   [junit4] <<< JVM J0: EOF ----

[...truncated 13 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:747: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:684: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/common-build.xml:529: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/common-build.xml:1457: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/common-build.xml:1014: There were test failures: 582 suites (10 ignored), 2331 tests, 3 errors, 1 failure, 32 ignored (16 assumptions) [seed: AB4EF3E1B06AF31B]

Total time: 275 minutes 39 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
No prior successful build to compare, so performing full copy of artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-NightlyTests-6.x - Build # 21 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-6.x/21/

2 tests failed.
FAILED:  org.apache.solr.cloud.DistribDocExpirationUpdateProcessorTest.test

Error Message:
Exactly one shard should have changed, instead: [shard2, shard1] nodes=([core_node3(shard2), core_node2(shard1), core_node4(shard1)]) expected:<1> but was:<2>

Stack Trace:
java.lang.AssertionError: Exactly one shard should have changed, instead: [shard2, shard1] nodes=([core_node3(shard2), core_node2(shard1), core_node4(shard1)]) expected:<1> but was:<2>
	at __randomizedtesting.SeedInfo.seed([89084DC3E22E3B:88DD37976D1E43C3]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.failNotEquals(Assert.java:647)
	at org.junit.Assert.assertEquals(Assert.java:128)
	at org.junit.Assert.assertEquals(Assert.java:472)
	at org.apache.solr.cloud.DistribDocExpirationUpdateProcessorTest.test(DistribDocExpirationUpdateProcessorTest.java:118)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:996)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:971)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.handler.TestReplicationHandler.doTestStressReplication

Error Message:
timed out waiting for collection1 startAt time to exceed: Thu Mar 24 08:02:17 CET 2016

Stack Trace:
java.lang.AssertionError: timed out waiting for collection1 startAt time to exceed: Thu Mar 24 08:02:17 CET 2016
	at __randomizedtesting.SeedInfo.seed([89084DC3E22E3B:DB22088BC6CA4788]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.solr.handler.TestReplicationHandler.watchCoreStartAt(TestReplicationHandler.java:1422)
	at org.apache.solr.handler.TestReplicationHandler.doTestStressReplication(TestReplicationHandler.java:774)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 11661 lines...]
   [junit4] Suite: org.apache.solr.cloud.DistribDocExpirationUpdateProcessorTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.DistribDocExpirationUpdateProcessorTest_89084DC3E22E3B-001/init-core-data-001
   [junit4]   2> 1325594 INFO  (SUITE-DistribDocExpirationUpdateProcessorTest-seed#[89084DC3E22E3B]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true)
   [junit4]   2> 1325594 INFO  (SUITE-DistribDocExpirationUpdateProcessorTest-seed#[89084DC3E22E3B]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   2> 1325597 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 1325598 INFO  (Thread-11720) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1325598 INFO  (Thread-11720) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 1325698 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [    ] o.a.s.c.ZkTestServer start zk server on port:41242
   [junit4]   2> 1325698 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1325698 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1325700 INFO  (zkCallback-1419-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@300fa29a name:ZooKeeperConnection Watcher:127.0.0.1:41242 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1325700 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1325701 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1325701 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 1325703 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1325703 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1325704 INFO  (zkCallback-1420-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2a5aeb40 name:ZooKeeperConnection Watcher:127.0.0.1:41242/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1325704 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1325704 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1325704 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 1325705 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 1325707 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 1325708 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 1325709 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-doc-expire-update-processor.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 1325709 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 1325711 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
   [junit4]   2> 1325711 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 1325713 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1325713 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1325715 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 1325715 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 1325716 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 1325716 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 1325717 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 1325717 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 1325719 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 1325719 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 1325720 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1325720 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1325722 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1325722 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1325723 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 1325723 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 1325725 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 1325725 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 1325812 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.DistribDocExpirationUpdateProcessorTest_89084DC3E22E3B-001/control-001/cores/collection1
   [junit4]   2> 1325814 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [    ] o.e.j.s.Server jetty-9.3.8.v20160314
   [junit4]   2> 1325815 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@60e252df{/,null,AVAILABLE}
   [junit4]   2> 1325817 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [    ] o.e.j.s.ServerConnector Started ServerConnector@5f78e424{HTTP/1.1,[http/1.1]}{127.0.0.1:50594}
   [junit4]   2> 1325817 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [    ] o.e.j.s.Server Started @1328572ms
   [junit4]   2> 1325817 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.DistribDocExpirationUpdateProcessorTest_89084DC3E22E3B-001/tempDir-001/control/data, hostContext=/, hostPort=50594, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.DistribDocExpirationUpdateProcessorTest_89084DC3E22E3B-001/control-001/cores}
   [junit4]   2> 1325818 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@73d16e93
   [junit4]   2> 1325818 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.DistribDocExpirationUpdateProcessorTest_89084DC3E22E3B-001/control-001'
   [junit4]   2> 1325818 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1325818 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1325818 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1325823 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1325825 INFO  (zkCallback-1421-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@222063af name:ZooKeeperConnection Watcher:127.0.0.1:41242/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1325825 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1325825 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1325826 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1325826 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.DistribDocExpirationUpdateProcessorTest_89084DC3E22E3B-001/control-001/solr.xml
   [junit4]   2> 1325831 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.DistribDocExpirationUpdateProcessorTest_89084DC3E22E3B-001/control-001/cores
   [junit4]   2> 1325831 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [    ] o.a.s.c.CoreContainer New CoreContainer 552428901
   [junit4]   2> 1325831 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.DistribDocExpirationUpdateProcessorTest_89084DC3E22E3B-001/control-001]
   [junit4]   2> 1325831 WARN  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.DistribDocExpirationUpdateProcessorTest_89084DC3E22E3B-001/control-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.DistribDocExpirationUpdateProcessorTest_89084DC3E22E3B-001/control-001/lib
   [junit4]   2> 1325832 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1325833 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1325834 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1325834 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1325834 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:41242/solr
   [junit4]   2> 1325834 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1325834 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1325835 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1325836 INFO  (zkCallback-1424-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1f66a291 name:ZooKeeperConnection Watcher:127.0.0.1:41242 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1325837 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1325837 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1325839 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [n:127.0.0.1:50594_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1325839 INFO  (zkCallback-1425-thread-1-processing-n:127.0.0.1:50594_) [n:127.0.0.1:50594_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3cfe7fe3 name:ZooKeeperConnection Watcher:127.0.0.1:41242/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1325839 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [n:127.0.0.1:50594_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1325840 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [n:127.0.0.1:50594_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 1325842 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [n:127.0.0.1:50594_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 1325844 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [n:127.0.0.1:50594_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 1325846 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [n:127.0.0.1:50594_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 1325847 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [n:127.0.0.1:50594_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 1325849 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [n:127.0.0.1:50594_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 1325850 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [n:127.0.0.1:50594_    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 1325851 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [n:127.0.0.1:50594_    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 1325852 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [n:127.0.0.1:50594_    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 1325853 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [n:127.0.0.1:50594_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1325854 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [n:127.0.0.1:50594_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (0)
   [junit4]   2> 1325856 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [n:127.0.0.1:50594_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 1325857 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [n:127.0.0.1:50594_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 1325858 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [n:127.0.0.1:50594_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1325858 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [n:127.0.0.1:50594_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/95604029526376452-127.0.0.1:50594_-n_0000000000
   [junit4]   2> 1325859 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [n:127.0.0.1:50594_    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:50594_
   [junit4]   2> 1325859 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [n:127.0.0.1:50594_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 1325860 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [n:127.0.0.1:50594_    ] o.a.s.c.Overseer Overseer (id=95604029526376452-127.0.0.1:50594_-n_0000000000) starting
   [junit4]   2> 1325861 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [n:127.0.0.1:50594_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 1325866 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [n:127.0.0.1:50594_    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 1325866 INFO  (OverseerCollectionConfigSetProcessor-95604029526376452-127.0.0.1:50594_-n_0000000000) [n:127.0.0.1:50594_    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 1325866 INFO  (OverseerStateUpdate-95604029526376452-127.0.0.1:50594_-n_0000000000) [n:127.0.0.1:50594_    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 1325867 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [n:127.0.0.1:50594_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50594_
   [junit4]   2> 1325868 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [n:127.0.0.1:50594_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:50594_
   [junit4]   2> 1325872 INFO  (OverseerStateUpdate-95604029526376452-127.0.0.1:50594_-n_0000000000) [n:127.0.0.1:50594_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (0)
   [junit4]   2> 1325872 INFO  (zkCallback-1425-thread-1-processing-n:127.0.0.1:50594_) [n:127.0.0.1:50594_    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [0])
   [junit4]   2> 1325873 INFO  (zkCallback-1425-thread-1-processing-n:127.0.0.1:50594_) [n:127.0.0.1:50594_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1325889 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [n:127.0.0.1:50594_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1325889 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [n:127.0.0.1:50594_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1325890 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [n:127.0.0.1:50594_    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.DistribDocExpirationUpdateProcessorTest_89084DC3E22E3B-001/control-001/cores
   [junit4]   2> 1325890 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [n:127.0.0.1:50594_    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, loadOnStartup=true, configSetProperties=configsetprops.json, collection=control_collection, coreNodeName=, dataDir=data/, shard=}
   [junit4]   2> 1325890 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [n:127.0.0.1:50594_    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.DistribDocExpirationUpdateProcessorTest_89084DC3E22E3B-001/control-001/cores/collection1
   [junit4]   2> 1325890 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [n:127.0.0.1:50594_    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 1325891 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [n:127.0.0.1:50594_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1
   [junit4]   2> 1325891 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [n:127.0.0.1:50594_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1325891 INFO  (coreLoadExecutor-3379-thread-1-processing-n:127.0.0.1:50594_) [n:127.0.0.1:50594_ c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 1325892 INFO  (coreLoadExecutor-3379-thread-1-processing-n:127.0.0.1:50594_) [n:127.0.0.1:50594_ c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1325893 INFO  (OverseerStateUpdate-95604029526376452-127.0.0.1:50594_-n_0000000000) [n:127.0.0.1:50594_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:50594",
   [junit4]   2>   "node_name":"127.0.0.1:50594_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 1325893 INFO  (OverseerStateUpdate-95604029526376452-127.0.0.1:50594_-n_0000000000) [n:127.0.0.1:50594_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:50594",
   [junit4]   2>   "node_name":"127.0.0.1:50594_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 1325893 INFO  (coreLoadExecutor-3379-thread-1-processing-n:127.0.0.1:50594_) [n:127.0.0.1:50594_ c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 1325894 INFO  (OverseerStateUpdate-95604029526376452-127.0.0.1:50594_-n_0000000000) [n:127.0.0.1:50594_    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 1325894 INFO  (OverseerStateUpdate-95604029526376452-127.0.0.1:50594_-n_0000000000) [n:127.0.0.1:50594_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 1325895 INFO  (zkCallback-1425-thread-1-processing-n:127.0.0.1:50594_) [n:127.0.0.1:50594_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1326894 INFO  (coreLoadExecutor-3379-thread-1-processing-n:127.0.0.1:50594_) [n:127.0.0.1:50594_ c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1326894 INFO  (coreLoadExecutor-3379-thread-1-processing-n:127.0.0.1:50594_) [n:127.0.0.1:50594_ c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 1326895 INFO  (coreLoadExecutor-3379-thread-1-processing-n:127.0.0.1:50594_) [n:127.0.0.1:50594_ c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 1326895 INFO  (coreLoadExecutor-3379-thread-1-processing-n:127.0.0.1:50594_) [n:127.0.0.1:50594_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/control_collection]
   [junit4]   2> 1326896 INFO  (coreLoadExecutor-3379-thread-1-processing-n:127.0.0.1:50594_) [n:127.0.0.1:50594_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/control_collection] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 1326896 INFO  (coreLoadExecutor-3379-thread-1-processing-n:127.0.0.1:50594_) [n:127.0.0.1:50594_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.DistribDocExpirationUpdateProcessorTest_89084DC3E22E3B-001/control-001/cores/collection1'
   [junit4]   2> 1326896 INFO  (coreLoadExecutor-3379-thread-1-processing-n:127.0.0.1:50594_) [n:127.0.0.1:50594_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1326896 INFO  (coreLoadExecutor-3379-thread-1-processing-n:127.0.0.1:50594_) [n:127.0.0.1:50594_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1326898 INFO  (coreLoadExecutor-3379-thread-1-processing-n:127.0.0.1:50594_) [n:127.0.0.1:50594_ c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 1326903 INFO  (coreLoadExecutor-3379-thread-1-processing-n:127.0.0.1:50594_) [n:127.0.0.1:50594_ c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 1326906 INFO  (coreLoadExecutor-3379-thread-1-processing-n:127.0.0.1:50594_) [n:127.0.0.1:50594_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.1.0
   [junit4]   2> 1326935 INFO  (coreLoadExecutor-3379-thread-1-processing-n:127.0.0.1:50594_) [n:127.0.0.1:50594_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1326940 INFO  (coreLoadExecutor-3379-thread-1-processing-n:127.0.0.1:50594_) [n:127.0.0.1:50594_ c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1327015 INFO  (coreLoadExecutor-3379-thread-1-processing-n:127.0.0.1:50594_) [n:127.0.0.1:50594_ c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 1327016 INFO  (coreLoadExecutor-3379-thread-1-processing-n:127.0.0.1:50594_) [n:127.0.0.1:50594_ c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 1327017 INFO  (coreLoadExecutor-3379-thread-1-processing-n:127.0.0.1:50594_) [n:127.0.0.1:50594_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1327019 INFO  (coreLoadExecutor-3379-thread-1-processing-n:127.0.0.1:50594_) [n:127.0.0.1:50594_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1327032 INFO  (coreLoadExecutor-3379-thread-1-processing-n:127.0.0.1:50594_) [n:127.0.0.1:50594_ c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1
   [junit4]   2> 1327032 INFO  (coreLoadExecutor-3379-thread-1-processing-n:127.0.0.1:50594_) [n:127.0.0.1:50594_ c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 1327032 INFO  (coreLoadExecutor-3379-thread-1-processing-n:127.0.0.1:50594_) [n:127.0.0.1:50594_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 1327033 INFO  (coreLoadExecutor-3379-thread-1-processing-n:127.0.0.1:50594_) [n:127.0.0.1:50594_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.DistribDocExpirationUpdateProcessorTest_89084DC3E22E3B-001/control-001/cores/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.DistribDocExpirationUpdateProcessorTest_89084DC3E22E3B-001/control-001/cores/collection1/data/]
   [junit4]   2> 1327033 INFO  (coreLoadExecutor-3379-thread-1-processing-n:127.0.0.1:50594_) [n:127.0.0.1:50594_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore JMX monitoring not detected for core: collection1
   [junit4]   2> 1327033 INFO  (coreLoadExecutor-3379-thread-1-processing-n:127.0.0.1:50594_) [n:127.0.0.1:50594_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.DistribDocExpirationUpdateProcessorTest_89084DC3E22E3B-001/control-001/cores/collection1/data
   [junit4]   2> 1327033 INFO  (coreLoadExecutor-3379-thread-1-processing-n:127.0.0.1:50594_) [n:127.0.0.1:50594_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.DistribDocExpirationUpdateProcessorTest_89084DC3E22E3B-001/control-001/cores/collection1/data/index/
   [junit4]   2> 1327033 WARN  (coreLoadExecutor-3379-thread-1-processing-n:127.0.0.1:50594_) [n:127.0.0.1:50594_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.DistribDocExpirationUpdateProcessorTest_89084DC3E22E3B-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 1327034 INFO  (coreLoadExecutor-3379-thread-1-processing-n:127.0.0.1:50594_) [n:127.0.0.1:50594_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.DistribDocExpirationUpdateProcessorTest_89084DC3E22E3B-001/control-001/cores/collection1/data/index
   [junit4]   2> 1327034 INFO  (coreLoadExecutor-3379-thread-1-processing-n:127.0.0.1:50594_) [n:127.0.0.1:50594_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=8, maxMergeAtOnceExplicit=4, maxMergedSegmentMB=4.8896484375, floorSegmentMB=0.8310546875, forceMergeDeletesPctAllowed=24.16925454222616, segmentsPerTier=29.0, maxCFSSegmentSizeMB=0.560546875, noCFSRatio=1.0
   [junit4]   2> 1327034 INFO  (coreLoadExecutor-3379-thread-1-processing-n:127.0.0.1:50594_) [n:127.0.0.1:50594_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@5a91bdf9 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@4ab563ad),segFN=segments_1,generation=1}
   [junit4]   2> 1327034 INFO  (coreLoadExecutor-3379-thread-1-processing-n:127.0.0.1:50594_) [n:127.0.0.1:50594_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 1327036 INFO  (coreLoadExecutor-3379-thread-1-processing-n:127.0.0.1:50594_) [n:127.0.0.1:50594_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "convert-ttl-defaults"
   [junit4]   2> 1327050 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:50594_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.DistribDocExpirationUpdateProcessorTest_89084DC3E22E3B-001/control-001/cores/collection1/data/
   [junit4]   2> 1327050 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:50594_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.DistribDocExpirationUpdateProcessorTest_89084DC3E22E3B-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 1327059 INFO  (coreLoadExecutor-3379-thread-1-processing-n:127.0.0.1:50594_) [n:127.0.0.1:50594_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "convert-ttl-field"
   [junit4]   2> 1327060 INFO  (coreLoadExecutor-3379-thread-1-processing-n:127.0.0.1:50594_) [n:127.0.0.1:50594_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "convert-ttl-param"
   [junit4]   2> 1327060 INFO  (coreLoadExecutor-3379-thread-1-processing-n:127.0.0.1:50594_) [n:127.0.0.1:50594_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "convert-ttl-field-with-param-default"
   [junit4]   2> 1327060 INFO  (coreLoadExecutor-3379-thread-1-processing-n:127.0.0.1:50594_) [n:127.0.0.1:50594_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "scheduled-delete" (default)
   [junit4]   2> 1327079 INFO  (coreLoadExecutor-3379-thread-1-processing-n:127.0.0.1:50594_) [n:127.0.0.1:50594_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "scheduled-delete" (default)
   [junit4]   2> 1327081 INFO  (coreLoadExecutor-3379-thread-1-processing-n:127.0.0.1:50594_) [n:127.0.0.1:50594_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1327082 INFO  (coreLoadExecutor-3379-thread-1-processing-n:127.0.0.1:50594_) [n:127.0.0.1:50594_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1327083 INFO  (coreLoadExecutor-3379-thread-1-processing-n:127.0.0.1:50594_) [n:127.0.0.1:50594_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1327085 INFO  (coreLoadExecutor-3379-thread-1-processing-n:127.0.0.1:50594_) [n:127.0.0.1:50594_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1327087 INFO  (coreLoadExecutor-3379-thread-1-processing-n:127.0.0.1:50594_) [n:127.0.0.1:50594_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/mbeans,,/update/csv,/sql,/update/json/docs,/admin/luke,/export,/select,/admin/segments,/get,/admin/system,/replication,/admin/properties,/config,/stream,/schema,/admin/plugins,/admin/logging,/update/json,/admin/ping,/admin/threads,/update,/admin/file
   [junit4]   2> 1327087 INFO  (coreLoadExecutor-3379-thread-1-processing-n:127.0.0.1:50594_) [n:127.0.0.1:50594_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 1327087 INFO  (coreLoadExecutor-3379-thread-1-processing-n:127.0.0.1:50594_) [n:127.0.0.1:50594_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1327087 INFO  (coreLoadExecutor-3379-thread-1-processing-n:127.0.0.1:50594_) [n:127.0.0.1:50594_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1327088 INFO  (coreLoadExecutor-3379-thread-1-processing-n:127.0.0.1:50594_) [n:127.0.0.1:50594_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1327088 INFO  (coreLoadExecutor-3379-thread-1-processing-n:127.0.0.1:50594_) [n:127.0.0.1:50594_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1327088 INFO  (coreLoadExecutor-3379-thread-1-processing-n:127.0.0.1:50594_) [n:127.0.0.1:50594_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=48, maxMergeAtOnceExplicit=23, maxMergedSegmentMB=76.306640625, floorSegmentMB=1.8828125, forceMergeDeletesPctAllowed=28.506955584445947, segmentsPerTier=16.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 1327095 INFO  (coreLoadExecutor-3379-thread-1-processing-n:127.0.0.1:50594_) [n:127.0.0.1:50594_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@5a91bdf9 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@4ab563ad),segFN=segments_1,generation=1}
   [junit4]   2> 1327095 INFO  (coreLoadExecutor-3379-thread-1-processing-n:127.0.0.1:50594_) [n:127.0.0.1:50594_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 1327095 INFO  (coreLoadExecutor-3379-thread-1-processing-n:127.0.0.1:50594_) [n:127.0.0.1:50594_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@229c3cd5[collection1] main]
   [junit4]   2> 1327095 INFO  (coreLoadExecutor-3379-thread-1-processing-n:127.0.0.1:50594_) [n:127.0.0.1:50594_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/control_collection]
   [junit4]   2> 1327097 INFO  (coreLoadExecutor-3379-thread-1-processing-n:127.0.0.1:50594_) [n:127.0.0.1:50594_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/control_collection] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 1327097 INFO  (coreLoadExecutor-3379-thread-1-processing-n:127.0.0.1:50594_) [n:127.0.0.1:50594_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 1327097 INFO  (coreLoadExecutor-3379-thread-1-processing-n:127.0.0.1:50594_) [n:127.0.0.1:50594_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1327097 INFO  (coreLoadExecutor-3379-thread-1-processing-n:127.0.0.1:50594_) [n:127.0.0.1:50594_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 1327097 INFO  (coreLoadExecutor-3379-thread-1-processing-n:127.0.0.1:50594_) [n:127.0.0.1:50594_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1327098 INFO  (coreLoadExecutor-3379-thread-1-processing-n:127.0.0.1:50594_) [n:127.0.0.1:50594_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 1327098 INFO  (coreLoadExecutor-3379-thread-1-processing-n:127.0.0.1:50594_) [n:127.0.0.1:50594_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1327098 INFO  (coreLoadExecutor-3379-thread-1-processing-n:127.0.0.1:50594_) [n:127.0.0.1:50594_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 1327116 INFO  (coreLoadExecutor-3379-thread-1-processing-n:127.0.0.1:50594_) [n:127.0.0.1:50594_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1327116 INFO  (coreLoadExecutor-3379-thread-1-processing-n:127.0.0.1:50594_) [n:127.0.0.1:50594_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 1327116 INFO  (coreLoadExecutor-3379-thread-1-processing-n:127.0.0.1:50594_) [n:127.0.0.1:50594_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 1327116 INFO  (coreLoadExecutor-3379-thread-1-processing-n:127.0.0.1:50594_) [n:127.0.0.1:50594_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 1327116 INFO  (coreLoadExecutor-3379-thread-1-processing-n:127.0.0.1:50594_) [n:127.0.0.1:50594_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1529664474012712960
   [junit4]   2> 1327116 INFO  (searcherExecutor-3380-thread-1-processing-n:127.0.0.1:50594_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:50594_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@229c3cd5[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1327118 INFO  (coreLoadExecutor-3379-thread-1-processing-n:127.0.0.1:50594_) [n:127.0.0.1:50594_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 2.0ms to seed version buckets with highest version 1529664474012712960
   [junit4]   2> 1327118 INFO  (coreLoadExecutor-3379-thread-1-processing-n:127.0.0.1:50594_) [n:127.0.0.1:50594_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 1327119 INFO  (coreLoadExecutor-3379-thread-1-processing-n:127.0.0.1:50594_) [n:127.0.0.1:50594_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 1327120 INFO  (coreZkRegister-3373-thread-1-processing-n:127.0.0.1:50594_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:50594_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:50594 collection:control_collection shard:shard1
   [junit4]   2> 1327120 INFO  (coreZkRegister-3373-thread-1-processing-n:127.0.0.1:50594_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:50594_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 1327124 INFO  (coreZkRegister-3373-thread-1-processing-n:127.0.0.1:50594_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:50594_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/95604029526376452-core_node1-n_0000000000
   [junit4]   2> 1327124 INFO  (coreZkRegister-3373-thread-1-processing-n:127.0.0.1:50594_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:50594_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard=shard1 and weAreReplacement=false and leaderVoteWait=10000
   [junit4]   2> 1327127 INFO  (OverseerStateUpdate-95604029526376452-127.0.0.1:50594_-n_0000000000) [n:127.0.0.1:50594_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 1327127 INFO  (coreZkRegister-3373-thread-1-processing-n:127.0.0.1:50594_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:50594_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1327127 INFO  (coreZkRegister-3373-thread-1-processing-n:127.0.0.1:50594_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:50594_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1327127 INFO  (coreZkRegister-3373-thread-1-processing-n:127.0.0.1:50594_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:50594_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:50594/collection1/
   [junit4]   2> 1327127 INFO  (coreZkRegister-3373-thread-1-processing-n:127.0.0.1:50594_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:50594_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 1327127 INFO  (coreZkRegister-3373-thread-1-processing-n:127.0.0.1:50594_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:50594_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:50594/collection1/ has no replicas
   [junit4]   2> 1327129 INFO  (coreZkRegister-3373-thread-1-processing-n:127.0.0.1:50594_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:50594_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 1327131 INFO  (coreZkRegister-3373-thread-1-processing-n:127.0.0.1:50594_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:50594_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/control_collection/leaders/shard1/leader after winning as /collections/control_collection/leader_elect/shard1/election/95604029526376452-core_node1-n_0000000000
   [junit4]   2> 1327132 INFO  (coreZkRegister-3373-thread-1-processing-n:127.0.0.1:50594_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:50594_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:50594/collection1/ shard1
   [junit4]   2> 1327133 INFO  (OverseerStateUpdate-95604029526376452-127.0.0.1:50594_-n_0000000000) [n:127.0.0.1:50594_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:50594",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 1
   [junit4]   2> 1327235 INFO  (zkCallback-1425-thread-1-processing-n:127.0.0.1:50594_) [n:127.0.0.1:50594_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1327284 INFO  (coreZkRegister-3373-thread-1-processing-n:127.0.0.1:50594_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:50594_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:50594/collection1/ and leader is http://127.0.0.1:50594/collection1/
   [junit4]   2> 1327285 INFO  (coreZkRegister-3373-thread-1-processing-n:127.0.0.1:50594_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:50594_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:50594
   [junit4]   2> 1327285 INFO  (coreZkRegister-3373-thread-1-processing-n:127.0.0.1:50594_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:50594_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1327285 INFO  (coreZkRegister-3373-thread-1-processing-n:127.0.0.1:50594_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:50594_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 1327285 INFO  (coreZkRegister-3373-thread-1-processing-n:127.0.0.1:50594_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:50594_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1327290 INFO  (OverseerStateUpdate-95604029526376452-127.0.0.1:50594_-n_0000000000) [n:127.0.0.1:50594_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:50594",
   [junit4]   2>   "node_name":"127.0.0.1:50594_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"} current state version: 2
   [junit4]   2> 1327291 INFO  (OverseerStateUpdate-95604029526376452-127.0.0.1:50594_-n_0000000000) [n:127.0.0.1:50594_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:50594",
   [junit4]   2>   "node_name":"127.0.0.1:50594_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 1327394 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1327396 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1327396 INFO  (zkCallback-1425-thread-1-processing-n:127.0.0.1:50594_) [n:127.0.0.1:50594_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1327396 INFO  (zkCallback-1427-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@615b8eac name:ZooKeeperConnection Watcher:127.0.0.1:41242/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1327397 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1327397 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1327397 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1327399 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1327400 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 1327535 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.DistribDocExpirationUpdateProcessorTest_89084DC3E22E3B-001/shard-1-001/cores/collection1
   [junit4]   2> 1327536 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.DistribDocExpirationUpdateProcessorTest_89084DC3E22E3B-001/shard-1-001
   [junit4]   2> 1327537 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [    ] o.e.j.s.Server jetty-9.3.8.v20160314
   [junit4]   2> 1327538 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@746cfbce{/,null,AVAILABLE}
   [junit4]   2> 1327538 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [    ] o.e.j.s.ServerConnector Started ServerConnector@6147460{HTTP/1.1,[http/1.1]}{127.0.0.1:46690}
   [junit4]   2> 1327539 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [    ] o.e.j.s.Server Started @1330294ms
   [junit4]   2> 1327539 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.DistribDocExpirationUpdateProcessorTest_89084DC3E22E3B-001/tempDir-001/jetty1, solrconfig=solrconfig.xml, hostContext=/, hostPort=46690, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.DistribDocExpirationUpdateProcessorTest_89084DC3E22E3B-001/shard-1-001/cores}
   [junit4]   2> 1327539 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@73d16e93
   [junit4]   2> 1327539 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.DistribDocExpirationUpdateProcessorTest_89084DC3E22E3B-001/shard-1-001'
   [junit4]   2> 1327539 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1327539 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1327539 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1327543 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1327549 INFO  (zkCallback-1428-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7ce3e4dc name:ZooKeeperConnection Watcher:127.0.0.1:41242/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1327549 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1327550 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1327551 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1327551 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.DistribDocExpirationUpdateProcessorTest_89084DC3E22E3B-001/shard-1-001/solr.xml
   [junit4]   2> 1327556 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.DistribDocExpirationUpdateProcessorTest_89084DC3E22E3B-001/shard-1-001/cores
   [junit4]   2> 1327556 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [    ] o.a.s.c.CoreContainer New CoreContainer 681816608
   [junit4]   2> 1327556 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.DistribDocExpirationUpdateProcessorTest_89084DC3E22E3B-001/shard-1-001]
   [junit4]   2> 1327556 WARN  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.DistribDocExpirationUpdateProcessorTest_89084DC3E22E3B-001/shard-1-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.DistribDocExpirationUpdateProcessorTest_89084DC3E22E3B-001/shard-1-001/lib
   [junit4]   2> 1327556 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1327557 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1327558 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1327558 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1327558 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:41242/solr
   [junit4]   2> 1327558 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1327558 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1327564 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1327564 INFO  (zkCallback-1431-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@14861e4 name:ZooKeeperConnection Watcher:127.0.0.1:41242 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1327565 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1327565 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1327567 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [n:127.0.0.1:46690_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1327568 INFO  (zkCallback-1432-thread-1-processing-n:127.0.0.1:46690_) [n:127.0.0.1:46690_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@10d7f0fb name:ZooKeeperConnection Watcher:127.0.0.1:41242/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1327568 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [n:127.0.0.1:46690_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1327575 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [n:127.0.0.1:46690_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1327576 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [n:127.0.0.1:46690_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1327577 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [n:127.0.0.1:46690_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1327578 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [n:127.0.0.1:46690_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/95604029526376456-127.0.0.1:46690_-n_0000000001
   [junit4]   2> 1327578 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [n:127.0.0.1:46690_    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/95604029526376452-127.0.0.1:50594_-n_0000000000 to know if I could be the leader
   [junit4]   2> 1327579 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [n:127.0.0.1:46690_    ] o.a.s.c.ZkController Publish node=127.0.0.1:46690_ as DOWN
   [junit4]   2> 1327580 INFO  (OverseerStateUpdate-95604029526376452-127.0.0.1:50594_-n_0000000000) [n:127.0.0.1:50594_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"downnode",
   [junit4]   2>   "node_name":"127.0.0.1:46690_"} current state version: 3
   [junit4]   2> 1327581 INFO  (OverseerStateUpdate-95604029526376452-127.0.0.1:50594_-n_0000000000) [n:127.0.0.1:50594_    ] o.a.s.c.o.NodeMutator DownNode state invoked for node: 127.0.0.1:46690_
   [junit4]   2> 1327685 INFO  (zkCallback-1432-thread-1-processing-n:127.0.0.1:46690_) [n:127.0.0.1:46690_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1327685 INFO  (zkCallback-1427-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1327685 INFO  (zkCallback-1425-thread-1-processing-n:127.0.0.1:50594_) [n:127.0.0.1:50594_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1328581 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [n:127.0.0.1:46690_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:46690_
   [junit4]   2> 1328581 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [n:127.0.0.1:46690_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:46690_
   [junit4]   2> 1328582 INFO  (zkCallback-1427-thread-1) [    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1328582 INFO  (zkCallback-1425-thread-1-processing-n:127.0.0.1:50594_) [n:127.0.0.1:50594_    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1328582 INFO  (zkCallback-1425-thread-1-processing-n:127.0.0.1:50594_) [n:127.0.0.1:50594_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 1328583 INFO  (zkCallback-1427-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 1328583 INFO  (zkCallback-1432-thread-1-processing-n:127.0.0.1:46690_) [n:127.0.0.1:46690_    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1328585 INFO  (zkCallback-1432-thread-1-processing-n:127.0.0.1:46690_) [n:127.0.0.1:46690_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 1328591 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [n:127.0.0.1:46690_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1328591 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [n:127.0.0.1:46690_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1328592 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [n:127.0.0.1:46690_    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.DistribDocExpirationUpdateProcessorTest_89084DC3E22E3B-001/shard-1-001/cores
   [junit4]   2> 1328593 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [n:127.0.0.1:46690_    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, loadOnStartup=true, configSetProperties=configsetprops.json, collection=collection1, coreNodeName=, dataDir=data/, shard=}
   [junit4]   2> 1328593 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [n:127.0.0.1:46690_    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.DistribDocExpirationUpdateProcessorTest_89084DC3E22E3B-001/shard-1-001/cores/collection1
   [junit4]   2> 1328593 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [n:127.0.0.1:46690_    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 1328611 INFO  (coreLoadExecutor-3391-thread-1-processing-n:127.0.0.1:46690_) [n:127.0.0.1:46690_ c:collection1   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 1328611 INFO  (coreLoadExecutor-3391-thread-1-processing-n:127.0.0.1:46690_) [n:127.0.0.1:46690_ c:collection1   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1328613 INFO  (OverseerStateUpdate-95604029526376452-127.0.0.1:50594_-n_0000000000) [n:127.0.0.1:50594_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:46690",
   [junit4]   2>   "node_name":"127.0.0.1:46690_",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "operation":"state"} current state version: 4
   [junit4]   2> 1328613 INFO  (OverseerStateUpdate-95604029526376452-127.0.0.1:50594_-n_0000000000) [n:127.0.0.1:50594_    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:46690",
   [junit4]   2>   "node_name":"127.0.0.1:46690_",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 1328613 INFO  (OverseerStateUpdate-95604029526376452-127.0.0.1:50594_-n_0000000000) [n:127.0.0.1:50594_    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 1328613 INFO  (OverseerStateUpdate-95604029526376452-127.0.0.1:50594_-n_0000000000) [n:127.0.0.1:50594_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2
   [junit4]   2> 1328614 INFO  (coreLoadExecutor-3391-thread-1-processing-n:127.0.0.1:46690_) [n:127.0.0.1:46690_ c:collection1   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 1328627 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [n:127.0.0.1:46690_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1
   [junit4]   2> 1328627 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[89084DC3E22E3B]) [n:127.0.0.1:46690_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1328715 INFO  (zkCallback-1427-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1328715 INFO  (zkCallback-1425-thread-1-processing-n:127.0.0.1:50594_) [n:127.0.0.1:50594_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1328715 INFO  (zkCallback-1432-thread-1-processing-n:127.0.0.1:46690_) [n:127.0.0.1:46690_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1329614 INFO  (coreLoadExecutor-3391-thread-1-processing-n:127.0.0.1:46690_) [n:127.0.0.1:46690_ c:collection1   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1329615 INFO  (coreLoadExecutor-3391-thread-1-processing-n:127.0.0.1:46690_) [n:127.0.0.1:46690_ c:collection1   x:collection1] o.a.s.c.ZkController Check for collection zkNode:collection1
   [junit4]   2> 1329615 INFO  (coreLoadExecutor-3391-thread-1-processing-n:127.0.0.1:46690_) [n:127.0.0.1:46690_ c:collection1   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 1329615 INFO  (coreLoadExecutor-3391-thread-1-processing-n:127.0.0.1:46690_) [n:127.0.0.1:46690_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/collection1]
   [junit4]   2> 1329615 INFO  (coreLoadExecutor-3391-thread-1-processing-n:127.0.0.1:46690_) [n:127.0.0.1:46690_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/collection1] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 1329616 INFO  (coreLoadExecutor-3391-thread-1-processing-n:127.0.0.1:46690_) [n:127.0.0.1:46690_ c:collection1   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.DistribDocExpirationUpdateProcessorTest_89084DC3E22E3B-001/shard-1-001/cores/collection1'
   [junit4]   2> 1329616 INFO  (coreLoadExecutor-3391-thread-1-processing-n:127.0.0.1:46690_) [n:127.0.0.1:46690_ c:collection1   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1329616 INFO  (coreLoadExecutor-3391-thread-1-processing-n:127.0.0.1:46690_) [n:127.0.0.1:46690_ c:collection1   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1329617 INFO  (coreLoadExecutor-3391-thread-1-processing-n:127.0.0.1:46690_) [n:127.0.0.1:46690_ c:collection1   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 1329622 INFO  (coreLoadExecutor-3391-thread-1-processing-n:127.0.0.1:46690_) [n:127.0.0.1:46690_ c:collection1   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 1329626 INFO  (coreLoadExecutor-3391-thread-1-processing-n:127.0.0.1:46690_) [n:127.0.0.1:46690_ c:collection1   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.1.0
   [junit4]   2> 1329662 INFO  (coreLoadExecutor-3391-thread-1-processing-n:127.0.0.1:46690_) [n:127.0.0.1:46690_ c:collection1   x:collection1] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1329671 INFO  (coreLoadExecutor-3391-thread-1-processing-n:127.0.0.1:46690_) [n:127.0.0.1:46690_ c:collection1   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1329799 INFO  (coreLoadExecutor-3391-thread-1-processing-n:127.0.0.1:46690_) [n:127.0.0.1:46690_ c:collection1   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 1329800 INFO  (coreLoadExecutor-3391-thread-1-processing-n:127.0.0.1:46690_) [n:127.0.0.1:46690_ c:collection1   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2

[...truncated too long message...]

g-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-collective-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-event-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-exception-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-journal-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-normalization-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-operational-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-referral-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-schema-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-subtree-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-trigger-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-jdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-jdbm1-2.0.0-M2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-kerberos-codec-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-ldif-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-mavibot-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-protocol-kerberos-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-protocol-ldap-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-protocol-shared-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-xdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/api-all-1.0.0-M20.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/bcprov-jdk15-1.45.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/cglib-nodep-2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/commons-collections-3.2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/commons-math3-3.4.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/easymock-3.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/ehcache-core-2.4.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/hadoop-common-2.6.0-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/hadoop-hdfs-2.6.0-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/hadoop-minikdc-2.6.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jackson-annotations-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jackson-databind-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jersey-core-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jersey-server-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jetty-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jetty-util-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/mina-core-2.0.0-M5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/mockito-core-1.9.5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/objenesis-1.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/presto-parser-0.122.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/slice-0.10.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/analysis/icu/lucene-analyzers-icu-6.1.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/contrib/solr-analysis-extras/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/queryparser/classes/test:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/contrib/analysis-extras/lib/icu4j-56.1.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-launcher.jar:/x1/jenkins/.ant/lib/ivy-2.3.0.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-antlr.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-regexp.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-resolver.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-net.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jmf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-oro.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-xalan2.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-logging.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jsch.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit4.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-javamail.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jai.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jdepend.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bsf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-swing.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-netrexx.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bcel.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-testutil.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-log4j.jar:/x1/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.8/lib/tools.jar:/x1/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.3.2.jar com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/temp/junit4-J1-20160324_061951_651.events @/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/temp/junit4-J1-20160324_061951_651.suites -stdin
   [junit4] ERROR: JVM J1 ended with an exception: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1501)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:121)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:955)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:952)
   [junit4] 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4] 	at java.lang.Thread.run(Thread.java:745)

BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:747: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:684: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/common-build.xml:529: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/common-build.xml:1457: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/common-build.xml:1014: At least one slave process threw an exception, first: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.

Total time: 231 minutes 41 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
No prior successful build to compare, so performing full copy of artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any