You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2015/07/10 07:36:16 UTC

[JENKINS] Lucene-Solr-NightlyTests-trunk - Build # 735 - Still Failing

Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-trunk/735/

2 tests failed.
REGRESSION:  org.apache.solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest.test

Error Message:
shard4 is not consistent.  Got 74 from http://127.0.0.1:50144/collection1lastClient and got 38 from http://127.0.0.1:47734/collection1

Stack Trace:
java.lang.AssertionError: shard4 is not consistent.  Got 74 from http://127.0.0.1:50144/collection1lastClient and got 38 from http://127.0.0.1:47734/collection1
	at __randomizedtesting.SeedInfo.seed([D83020E37C7B824C:50641F39D287EFB4]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1244)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1223)
	at org.apache.solr.cloud.ChaosMonkeySafeLeaderTest.test(ChaosMonkeySafeLeaderTest.java:165)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:483)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:960)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:935)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=55133, name=collection5, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=55133, name=collection5, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:40332/zw: Could not find collection : awholynewstresscollection_collection5_1
	at __randomizedtesting.SeedInfo.seed([D83020E37C7B824C]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:560)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:234)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:226)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:376)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:328)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1086)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:856)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:799)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:894)




Build Log:
[...truncated 10145 lines...]
   [junit4] Suite: org.apache.solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_D83020E37C7B824C-001/init-core-data-001
   [junit4]   2> 311086 INFO  (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[D83020E37C7B824C]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true)
   [junit4]   2> 311097 INFO  (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[D83020E37C7B824C]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   1> Formatting using clusterid: testClusterID
   [junit4]   2> 311335 WARN  (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[D83020E37C7B824C]-worker) [    ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
   [junit4]   2> 311362 WARN  (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[D83020E37C7B824C]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 311395 INFO  (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[D83020E37C7B824C]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 311444 INFO  (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[D83020E37C7B824C]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/hdfs to ./temp/Jetty_localhost_33429_hdfs____gkbop1/webapp
   [junit4]   2> 311926 INFO  (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[D83020E37C7B824C]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 312925 INFO  (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[D83020E37C7B824C]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:33429
   [junit4]   2> 313412 WARN  (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[D83020E37C7B824C]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 313414 INFO  (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[D83020E37C7B824C]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 313433 INFO  (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[D83020E37C7B824C]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_43224_datanode____1jj798/webapp
   [junit4]   2> 313735 INFO  (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[D83020E37C7B824C]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 313862 ERROR (Finalizer) [    ] o.a.s.c.CoreContainer CoreContainer was not close prior to finalize(), indicates a bug -- POSSIBLE RESOURCE LEAK!!!  instance=381596602
   [junit4]   2> 313862 ERROR (Finalizer) [    ] o.a.s.c.CoreContainer CoreContainer was not close prior to finalize(), indicates a bug -- POSSIBLE RESOURCE LEAK!!!  instance=1924569860
   [junit4]   2> 313874 ERROR (Finalizer) [    ] o.a.s.c.CoreContainer CoreContainer was not close prior to finalize(), indicates a bug -- POSSIBLE RESOURCE LEAK!!!  instance=1230234716
   [junit4]   2> 313875 ERROR (Finalizer) [    ] o.a.s.c.CoreContainer CoreContainer was not close prior to finalize(), indicates a bug -- POSSIBLE RESOURCE LEAK!!!  instance=871379439
   [junit4]   2> 314483 INFO  (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[D83020E37C7B824C]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:43224
   [junit4]   2> 314897 WARN  (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[D83020E37C7B824C]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 314899 INFO  (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[D83020E37C7B824C]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 314929 INFO  (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[D83020E37C7B824C]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_59026_datanode____.f9v5qh/webapp
   [junit4]   2> 315036 INFO  (IPC Server handler 3 on 40080) [    ] BlockStateChange BLOCK* processReport: from storage DS-db8b2c25-0a98-41e8-acbf-1857fccc51e7 node DatanodeRegistration(127.0.0.1, datanodeUuid=5fab64e0-76d4-4df0-b82f-13b4bd7ee17b, infoPort=43224, ipcPort=41183, storageInfo=lv=-56;cid=testClusterID;nsid=431653412;c=0), blocks: 0, hasStaleStorages: true, processing time: 11 msecs
   [junit4]   2> 315037 INFO  (IPC Server handler 3 on 40080) [    ] BlockStateChange BLOCK* processReport: from storage DS-8e4988b7-1db8-4a4e-a693-b101aef6fc69 node DatanodeRegistration(127.0.0.1, datanodeUuid=5fab64e0-76d4-4df0-b82f-13b4bd7ee17b, infoPort=43224, ipcPort=41183, storageInfo=lv=-56;cid=testClusterID;nsid=431653412;c=0), blocks: 0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 315258 INFO  (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[D83020E37C7B824C]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 315940 INFO  (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[D83020E37C7B824C]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:59026
   [junit4]   2> 316419 INFO  (IPC Server handler 3 on 40080) [    ] BlockStateChange BLOCK* processReport: from storage DS-9f96c59c-16f6-4537-8b0c-3a54eaef5e2f node DatanodeRegistration(127.0.0.1, datanodeUuid=63077f5a-7a19-455d-81ef-2420041d2612, infoPort=59026, ipcPort=57273, storageInfo=lv=-56;cid=testClusterID;nsid=431653412;c=0), blocks: 0, hasStaleStorages: true, processing time: 0 msecs
   [junit4]   2> 316420 INFO  (IPC Server handler 3 on 40080) [    ] BlockStateChange BLOCK* processReport: from storage DS-daf67910-48e4-479c-a508-604bd2255d05 node DatanodeRegistration(127.0.0.1, datanodeUuid=63077f5a-7a19-455d-81ef-2420041d2612, infoPort=59026, ipcPort=57273, storageInfo=lv=-56;cid=testClusterID;nsid=431653412;c=0), blocks: 0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 316574 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 316589 INFO  (Thread-789) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 316590 INFO  (Thread-789) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 316757 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [    ] o.a.s.c.ZkTestServer start zk server on port:46868
   [junit4]   2> 316757 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 316777 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 316805 INFO  (zkCallback-114-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4ed325d2 name:ZooKeeperConnection Watcher:127.0.0.1:46868 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 316806 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 316806 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 316806 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 316825 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 316885 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 316921 INFO  (zkCallback-115-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@523fdd07 name:ZooKeeperConnection Watcher:127.0.0.1:46868/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 316922 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 316923 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 316923 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 316925 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 316927 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 316928 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 316930 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 316930 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 316933 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
   [junit4]   2> 316933 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 316935 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 316935 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 316943 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 316943 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 316945 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 316945 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 316947 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 316947 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 316950 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 316950 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 316952 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 316952 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 316953 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 316954 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 316955 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 316955 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 316957 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 316957 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 317762 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_D83020E37C7B824C-001/control-001/cores/collection1
   [junit4]   2> 317775 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [    ] o.e.j.s.Server jetty-9.2.11.v20150529
   [junit4]   2> 317859 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@335d8166{/,null,AVAILABLE}
   [junit4]   2> 317860 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [    ] o.e.j.s.ServerConnector Started ServerConnector@b3ebbb5{HTTP/1.1}{127.0.0.1:36183}
   [junit4]   2> 317860 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [    ] o.e.j.s.Server Started @321304ms
   [junit4]   2> 317860 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=hdfs://localhost:40080/hdfs__localhost_40080__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-trunk_solr_build_solr-core_test_J0_temp_solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_D83020E37C7B824C-001_tempDir-002_control_data, hostContext=/, hostPort=36183, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_D83020E37C7B824C-001/control-001/cores}
   [junit4]   2> 317861 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@c387f44
   [junit4]   2> 317861 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_D83020E37C7B824C-001/control-001/'
   [junit4]   2> 317877 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_D83020E37C7B824C-001/control-001/solr.xml
   [junit4]   2> 317910 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_D83020E37C7B824C-001/control-001/cores
   [junit4]   2> 317910 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [    ] o.a.s.c.CoreContainer New CoreContainer 984647594
   [junit4]   2> 317910 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_D83020E37C7B824C-001/control-001/]
   [junit4]   2> 317910 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_D83020E37C7B824C-001/control-001/lib
   [junit4]   2> 317910 WARN  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_D83020E37C7B824C-001/control-001/lib).
   [junit4]   2> 317929 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 317931 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 317931 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 317931 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 317931 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46868/solr
   [junit4]   2> 317932 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 317932 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 317942 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 317961 INFO  (zkCallback-117-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@54abc0b name:ZooKeeperConnection Watcher:127.0.0.1:46868 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 317962 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 317963 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 318013 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [n:127.0.0.1:36183_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 318039 INFO  (zkCallback-118-thread-1-processing-n:127.0.0.1:36183_) [n:127.0.0.1:36183_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1ab7257f name:ZooKeeperConnection Watcher:127.0.0.1:46868/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 318039 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [n:127.0.0.1:36183_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 318041 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [n:127.0.0.1:36183_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 318043 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [n:127.0.0.1:36183_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 318045 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [n:127.0.0.1:36183_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 318047 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [n:127.0.0.1:36183_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 318048 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [n:127.0.0.1:36183_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 318050 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [n:127.0.0.1:36183_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 318052 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [n:127.0.0.1:36183_    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 318053 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [n:127.0.0.1:36183_    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 318054 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [n:127.0.0.1:36183_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:36183_
   [junit4]   2> 318054 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [n:127.0.0.1:36183_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:36183_
   [junit4]   2> 318056 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [n:127.0.0.1:36183_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 318057 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [n:127.0.0.1:36183_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 318058 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [n:127.0.0.1:36183_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 318058 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [n:127.0.0.1:36183_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94142656949518339-127.0.0.1:36183_-n_0000000000
   [junit4]   2> 318059 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [n:127.0.0.1:36183_    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:36183_
   [junit4]   2> 318059 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [n:127.0.0.1:36183_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 318060 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [n:127.0.0.1:36183_    ] o.a.s.c.Overseer Overseer (id=94142656949518339-127.0.0.1:36183_-n_0000000000) starting
   [junit4]   2> 318062 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [n:127.0.0.1:36183_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 318075 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [n:127.0.0.1:36183_    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 318086 INFO  (OverseerStateUpdate-94142656949518339-127.0.0.1:36183_-n_0000000000) [n:127.0.0.1:36183_    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 318095 INFO  (OverseerCollectionProcessor-94142656949518339-127.0.0.1:36183_-n_0000000000) [n:127.0.0.1:36183_    ] o.a.s.c.OverseerCollectionProcessor Process current queue of collection creations
   [junit4]   2> 318103 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [n:127.0.0.1:36183_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 318106 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [n:127.0.0.1:36183_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 318106 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [n:127.0.0.1:36183_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 318107 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [n:127.0.0.1:36183_    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_D83020E37C7B824C-001/control-001/cores
   [junit4]   2> 318108 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [n:127.0.0.1:36183_    ] o.a.s.c.SolrCore Created CoreDescriptor: {name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, loadOnStartup=true, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_D83020E37C7B824C-001/control-001/cores/collection1, collection=control_collection, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_D83020E37C7B824C-001/control-001/cores/collection1/, coreNodeName=, dataDir=data/, shard=}
   [junit4]   2> 318108 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [n:127.0.0.1:36183_    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_D83020E37C7B824C-001/control-001/cores/collection1/
   [junit4]   2> 318108 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [n:127.0.0.1:36183_    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 318122 INFO  (coreLoadExecutor-622-thread-1-processing-n:127.0.0.1:36183_) [n:127.0.0.1:36183_ c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 318123 INFO  (coreLoadExecutor-622-thread-1-processing-n:127.0.0.1:36183_) [n:127.0.0.1:36183_ c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 318126 INFO  (zkCallback-118-thread-1-processing-n:127.0.0.1:36183_) [n:127.0.0.1:36183_    ] o.a.s.c.DistributedQueue NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 318126 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [n:127.0.0.1:36183_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0
   [junit4]   2> 318126 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [n:127.0.0.1:36183_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 318127 INFO  (OverseerStateUpdate-94142656949518339-127.0.0.1:36183_-n_0000000000) [n:127.0.0.1:36183_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:36183",
   [junit4]   2>   "node_name":"127.0.0.1:36183_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 318127 INFO  (OverseerStateUpdate-94142656949518339-127.0.0.1:36183_-n_0000000000) [n:127.0.0.1:36183_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:36183",
   [junit4]   2>   "node_name":"127.0.0.1:36183_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 318127 INFO  (OverseerStateUpdate-94142656949518339-127.0.0.1:36183_-n_0000000000) [n:127.0.0.1:36183_    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 318128 INFO  (OverseerStateUpdate-94142656949518339-127.0.0.1:36183_-n_0000000000) [n:127.0.0.1:36183_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 318129 INFO  (zkCallback-118-thread-1-processing-n:127.0.0.1:36183_) [n:127.0.0.1:36183_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 318131 INFO  (zkCallback-118-thread-1-processing-n:127.0.0.1:36183_) [n:127.0.0.1:36183_    ] o.a.s.c.c.ZkStateReader Updated cluster state version to 1
   [junit4]   2> 318188 INFO  (coreLoadExecutor-622-thread-1-processing-n:127.0.0.1:36183_) [n:127.0.0.1:36183_ c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 318189 INFO  (coreLoadExecutor-622-thread-1-processing-n:127.0.0.1:36183_) [n:127.0.0.1:36183_ c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 318189 INFO  (coreLoadExecutor-622-thread-1-processing-n:127.0.0.1:36183_) [n:127.0.0.1:36183_ c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 318191 INFO  (coreLoadExecutor-622-thread-1-processing-n:127.0.0.1:36183_) [n:127.0.0.1:36183_ c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 318191 INFO  (coreLoadExecutor-622-thread-1-processing-n:127.0.0.1:36183_) [n:127.0.0.1:36183_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 318199 INFO  (coreLoadExecutor-622-thread-1-processing-n:127.0.0.1:36183_) [n:127.0.0.1:36183_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 318199 INFO  (coreLoadExecutor-622-thread-1-processing-n:127.0.0.1:36183_) [n:127.0.0.1:36183_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_D83020E37C7B824C-001/control-001/cores/collection1/'
   [junit4]   2> 318225 INFO  (coreLoadExecutor-622-thread-1-processing-n:127.0.0.1:36183_) [n:127.0.0.1:36183_ c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 318242 INFO  (coreLoadExecutor-622-thread-1-processing-n:127.0.0.1:36183_) [n:127.0.0.1:36183_ c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 318246 INFO  (coreLoadExecutor-622-thread-1-processing-n:127.0.0.1:36183_) [n:127.0.0.1:36183_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.0.0
   [junit4]   2> 318294 INFO  (coreLoadExecutor-622-thread-1-processing-n:127.0.0.1:36183_) [n:127.0.0.1:36183_ c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 318295 INFO  (coreLoadExecutor-622-thread-1-processing-n:127.0.0.1:36183_) [n:127.0.0.1:36183_ c:control_collection   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 318310 INFO  (coreLoadExecutor-622-thread-1-processing-n:127.0.0.1:36183_) [n:127.0.0.1:36183_ c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 318419 INFO  (coreLoadExecutor-622-thread-1-processing-n:127.0.0.1:36183_) [n:127.0.0.1:36183_ c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 318420 INFO  (coreLoadExecutor-622-thread-1-processing-n:127.0.0.1:36183_) [n:127.0.0.1:36183_ c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 318428 INFO  (coreLoadExecutor-622-thread-1-processing-n:127.0.0.1:36183_) [n:127.0.0.1:36183_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 318436 INFO  (coreLoadExecutor-622-thread-1-processing-n:127.0.0.1:36183_) [n:127.0.0.1:36183_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 318461 INFO  (coreLoadExecutor-622-thread-1-processing-n:127.0.0.1:36183_) [n:127.0.0.1:36183_ c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 318462 INFO  (coreLoadExecutor-622-thread-1-processing-n:127.0.0.1:36183_) [n:127.0.0.1:36183_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.HdfsDirectoryFactory
   [junit4]   2> 318462 INFO  (coreLoadExecutor-622-thread-1-processing-n:127.0.0.1:36183_) [n:127.0.0.1:36183_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:40080/solr_hdfs_home
   [junit4]   2> 318462 INFO  (coreLoadExecutor-622-thread-1-processing-n:127.0.0.1:36183_) [n:127.0.0.1:36183_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
   [junit4]   2> 318462 INFO  (coreLoadExecutor-622-thread-1-processing-n:127.0.0.1:36183_) [n:127.0.0.1:36183_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_D83020E37C7B824C-001/control-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 318462 INFO  (coreLoadExecutor-622-thread-1-processing-n:127.0.0.1:36183_) [n:127.0.0.1:36183_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@49f78cd8
   [junit4]   2> 318463 INFO  (coreLoadExecutor-622-thread-1-processing-n:127.0.0.1:36183_) [n:127.0.0.1:36183_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:40080/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 318555 INFO  (coreLoadExecutor-622-thread-1-processing-n:127.0.0.1:36183_) [n:127.0.0.1:36183_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.h.HdfsLocalityReporter Registering direcotry hdfs://localhost:40080/solr_hdfs_home/control_collection/core_node1/data for locality metrics.
   [junit4]   2> 318556 INFO  (coreLoadExecutor-622-thread-1-processing-n:127.0.0.1:36183_) [n:127.0.0.1:36183_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:40080/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 318561 INFO  (coreLoadExecutor-622-thread-1-processing-n:127.0.0.1:36183_) [n:127.0.0.1:36183_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=hdfs://localhost:40080/solr_hdfs_home/control_collection/core_node1/data/index/
   [junit4]   2> 318843 WARN  (coreLoadExecutor-622-thread-1-processing-n:127.0.0.1:36183_) [n:127.0.0.1:36183_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory 'hdfs:/localhost:40080/solr_hdfs_home/control_collection/core_node1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 318843 INFO  (coreLoadExecutor-622-thread-1-processing-n:127.0.0.1:36183_) [n:127.0.0.1:36183_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:40080/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 318886 INFO  (coreLoadExecutor-622-thread-1-processing-n:127.0.0.1:36183_) [n:127.0.0.1:36183_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
   [junit4]   2> 318888 INFO  (coreLoadExecutor-622-thread-1-processing-n:127.0.0.1:36183_) [n:127.0.0.1:36183_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
   [junit4]   2> 318888 INFO  (coreLoadExecutor-622-thread-1-processing-n:127.0.0.1:36183_) [n:127.0.0.1:36183_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
   [junit4]   2> 318926 INFO  (coreLoadExecutor-622-thread-1-processing-n:127.0.0.1:36183_) [n:127.0.0.1:36183_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
   [junit4]   2> 318926 INFO  (coreLoadExecutor-622-thread-1-processing-n:127.0.0.1:36183_) [n:127.0.0.1:36183_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.h.HdfsLocalityReporter Registering direcotry hdfs://localhost:40080/solr_hdfs_home/control_collection/core_node1/data/index for locality metrics.
   [junit4]   2> 318929 INFO  (coreLoadExecutor-622-thread-1-processing-n:127.0.0.1:36183_) [n:127.0.0.1:36183_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:40080/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 318932 INFO  (coreLoadExecutor-622-thread-1-processing-n:127.0.0.1:36183_) [n:127.0.0.1:36183_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=9, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.5899791287170559]
   [junit4]   2> 319159 INFO  (IPC Server handler 0 on 40080) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:40521 is added to blk_1073741825_1001{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-db8b2c25-0a98-41e8-acbf-1857fccc51e7:NORMAL:127.0.0.1:34241|RBW], ReplicaUnderConstruction[[DISK]DS-9f96c59c-16f6-4537-8b0c-3a54eaef5e2f:NORMAL:127.0.0.1:40521|FINALIZED]]} size 0
   [junit4]   2> 319184 INFO  (IPC Server handler 1 on 40080) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:34241 is added to blk_1073741825_1001 size 71
   [junit4]   2> 319213 INFO  (coreLoadExecutor-622-thread-1-processing-n:127.0.0.1:36183_) [n:127.0.0.1:36183_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(BlockDirectory(HdfsDirectory@b6d144c6 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@533caba4); maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 319213 INFO  (coreLoadExecutor-622-thread-1-processing-n:127.0.0.1:36183_) [n:127.0.0.1:36183_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 319225 INFO  (coreLoadExecutor-622-thread-1-processing-n:127.0.0.1:36183_) [n:127.0.0.1:36183_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 319225 INFO  (coreLoadExecutor-622-thread-1-processing-n:127.0.0.1:36183_) [n:127.0.0.1:36183_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 319225 INFO  (coreLoadExecutor-622-thread-1-processing-n:127.0.0.1:36183_) [n:127.0.0.1:36183_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 319225 INFO  (coreLoadExecutor-622-thread-1-processing-n:127.0.0.1:36183_) [n:127.0.0.1:36183_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 319225 INFO  (coreLoadExecutor-622-thread-1-processing-n:127.0.0.1:36183_) [n:127.0.0.1:36183_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 319226 INFO  (coreLoadExecutor-622-thread-1-processing-n:127.0.0.1:36183_) [n:127.0.0.1:36183_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 319226 INFO  (coreLoadExecutor-622-thread-1-processing-n:127.0.0.1:36183_) [n:127.0.0.1:36183_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 319226 INFO  (coreLoadExecutor-622-thread-1-processing-n:127.0.0.1:36183_) [n:127.0.0.1:36183_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 319226 INFO  (coreLoadExecutor-622-thread-1-processing-n:127.0.0.1:36183_) [n:127.0.0.1:36183_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 319229 INFO  (coreLoadExecutor-622-thread-1-processing-n:127.0.0.1:36183_) [n:127.0.0.1:36183_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 319230 INFO  (coreLoadExecutor-622-thread-1-processing-n:127.0.0.1:36183_) [n:127.0.0.1:36183_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 319310 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:36183_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in hdfs://localhost:40080/solr_hdfs_home/control_collection/core_node1/data/
   [junit4]   2> 319423 INFO  (coreLoadExecutor-622-thread-1-processing-n:127.0.0.1:36183_) [n:127.0.0.1:36183_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 319424 INFO  (coreLoadExecutor-622-thread-1-processing-n:127.0.0.1:36183_) [n:127.0.0.1:36183_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 319450 INFO  (coreLoadExecutor-622-thread-1-processing-n:127.0.0.1:36183_) [n:127.0.0.1:36183_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/mbeans,standard,/update/csv,/update/json/docs,/admin/luke,/admin/segments,/get,/admin/system,/replication,/admin/properties,/config,/schema,/admin/plugins,/admin/logging,/update/json,/admin/threads,/admin/ping,/update,/admin/file
   [junit4]   2> 319450 INFO  (coreLoadExecutor-622-thread-1-processing-n:127.0.0.1:36183_) [n:127.0.0.1:36183_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 319483 INFO  (coreLoadExecutor-622-thread-1-processing-n:127.0.0.1:36183_) [n:127.0.0.1:36183_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
   [junit4]   2> 319483 INFO  (coreLoadExecutor-622-thread-1-processing-n:127.0.0.1:36183_) [n:127.0.0.1:36183_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing HdfsUpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 tlogDfsReplication=2
   [junit4]   2> 319526 INFO  (coreLoadExecutor-622-thread-1-processing-n:127.0.0.1:36183_) [n:127.0.0.1:36183_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 319526 INFO  (coreLoadExecutor-622-thread-1-processing-n:127.0.0.1:36183_) [n:127.0.0.1:36183_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 319530 INFO  (coreLoadExecutor-622-thread-1-processing-n:127.0.0.1:36183_) [n:127.0.0.1:36183_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=10, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
   [junit4]   2> 319784 INFO  (coreLoadExecutor-622-thread-1-processing-n:127.0.0.1:36183_) [n:127.0.0.1:36183_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(BlockDirectory(HdfsDirectory@b6d144c6 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@533caba4); maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 319785 INFO  (coreLoadExecutor-622-thread-1-processing-n:127.0.0.1:36183_) [n:127.0.0.1:36183_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 319785 INFO  (coreLoadExecutor-622-thread-1-processing-n:127.0.0.1:36183_) [n:127.0.0.1:36183_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@242aca3b[collection1] main
   [junit4]   2> 319785 INFO  (coreLoadExecutor-622-thread-1-processing-n:127.0.0.1:36183_) [n:127.0.0.1:36183_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 319786 INFO  (coreLoadExecutor-622-thread-1-processing-n:127.0.0.1:36183_) [n:127.0.0.1:36183_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 319787 INFO  (coreLoadExecutor-622-thread-1-processing-n:127.0.0.1:36183_) [n:127.0.0.1:36183_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 319787 INFO  (coreLoadExecutor-622-thread-1-processing-n:127.0.0.1:36183_) [n:127.0.0.1:36183_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 319787 INFO  (coreLoadExecutor-622-thread-1-processing-n:127.0.0.1:36183_) [n:127.0.0.1:36183_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 319787 INFO  (coreLoadExecutor-622-thread-1-processing-n:127.0.0.1:36183_) [n:127.0.0.1:36183_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 319789 INFO  (coreLoadExecutor-622-thread-1-processing-n:127.0.0.1:36183_) [n:127.0.0.1:36183_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 319789 INFO  (coreLoadExecutor-622-thread-1-processing-n:127.0.0.1:36183_) [n:127.0.0.1:36183_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 319789 INFO  (coreLoadExecutor-622-thread-1-processing-n:127.0.0.1:36183_) [n:127.0.0.1:36183_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 319790 INFO  (coreLoadExecutor-622-thread-1-processing-n:127.0.0.1:36183_) [n:127.0.0.1:36183_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 319790 INFO  (searcherExecutor-623-thread-1-processing-n:127.0.0.1:36183_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:36183_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@242aca3b[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 319827 WARN  (coreLoadExecutor-622-thread-1-processing-n:127.0.0.1:36183_) [n:127.0.0.1:36183_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.h.HdfsLocalityReporter Could not retrieve locality information for hdfs://localhost:60475/solr3 due to exception: java.net.ConnectException: Call From lucene1-us-west/10.41.0.5 to localhost:60475 failed on connection exception: java.net.ConnectException: Connection refused; For more details see:  http://wiki.apache.org/hadoop/ConnectionRefused
   [junit4]   2> 319845 INFO  (coreLoadExecutor-622-thread-1-processing-n:127.0.0.1:36183_) [n:127.0.0.1:36183_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 319845 INFO  (coreLoadExecutor-622-thread-1-processing-n:127.0.0.1:36183_) [n:127.0.0.1:36183_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 256 version buckets from index
   [junit4]   2> 319846 INFO  (coreLoadExecutor-622-thread-1-processing-n:127.0.0.1:36183_) [n:127.0.0.1:36183_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 319846 INFO  (coreLoadExecutor-622-thread-1-processing-n:127.0.0.1:36183_) [n:127.0.0.1:36183_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1506282514604359680
   [junit4]   2> 319846 INFO  (coreLoadExecutor-622-thread-1-processing-n:127.0.0.1:36183_) [n:127.0.0.1:36183_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 1 ms to seed version buckets with highest version 1506282514604359680
   [junit4]   2> 319846 INFO  (coreLoadExecutor-622-thread-1-processing-n:127.0.0.1:36183_) [n:127.0.0.1:36183_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 319847 INFO  (coreLoadExecutor-622-thread-1-processing-n:127.0.0.1:36183_) [n:127.0.0.1:36183_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 319856 INFO  (coreZkRegister-616-thread-1-processing-n:127.0.0.1:36183_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:36183_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:36183 collection:control_collection shard:shard1
   [junit4]   2> 319860 INFO  (coreZkRegister-616-thread-1-processing-n:127.0.0.1:36183_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:36183_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 319883 INFO  (coreZkRegister-616-thread-1-processing-n:127.0.0.1:36183_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:36183_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94142656949518339-core_node1-n_0000000000
   [junit4]   2> 319893 INFO  (coreZkRegister-616-thread-1-processing-n:127.0.0.1:36183_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:36183_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 319897 INFO  (zkCallback-118-thread-1-processing-n:127.0.0.1:36183_) [n:127.0.0.1:36183_    ] o.a.s.c.DistributedQueue NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 319898 INFO  (OverseerStateUpdate-94142656949518339-127.0.0.1:36183_-n_0000000000) [n:127.0.0.1:36183_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 319900 INFO  (zkCallback-118-thread-1-processing-n:127.0.0.1:36183_) [n:127.0.0.1:36183_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 319901 INFO  (zkCallback-118-thread-1-processing-n:127.0.0.1:36183_) [n:127.0.0.1:36183_    ] o.a.s.c.c.ZkStateReader Updated cluster state version to 2
   [junit4]   2> 319904 INFO  (coreZkRegister-616-thread-1-processing-n:127.0.0.1:36183_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:36183_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 319904 INFO  (coreZkRegister-616-thread-1-processing-n:127.0.0.1:36183_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:36183_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 319904 INFO  (coreZkRegister-616-thread-1-processing-n:127.0.0.1:36183_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:36183_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:36183/collection1/
   [junit4]   2> 319904 INFO  (coreZkRegister-616-thread-1-processing-n:127.0.0.1:36183_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:36183_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 319904 INFO  (coreZkRegister-616-thread-1-processing-n:127.0.0.1:36183_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:36183_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:36183/collection1/ has no replicas
   [junit4]   2> 319904 INFO  (coreZkRegister-616-thread-1-processing-n:127.0.0.1:36183_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:36183_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:36183/collection1/ shard1
   [junit4]   2> 319904 INFO  (coreZkRegister-616-thread-1-processing-n:127.0.0.1:36183_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:36183_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 319908 INFO  (zkCallback-118-thread-1-processing-n:127.0.0.1:36183_) [n:127.0.0.1:36183_    ] o.a.s.c.DistributedQueue NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 319909 INFO  (OverseerStateUpdate-94142656949518339-127.0.0.1:36183_-n_0000000000) [n:127.0.0.1:36183_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:36183",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 2
   [junit4]   2> 320012 INFO  (zkCallback-118-thread-1-processing-n:127.0.0.1:36183_) [n:127.0.0.1:36183_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 320014 INFO  (zkCallback-118-thread-1-processing-n:127.0.0.1:36183_) [n:127.0.0.1:36183_    ] o.a.s.c.c.ZkStateReader Updated cluster state version to 3
   [junit4]   2> 320065 INFO  (coreZkRegister-616-thread-1-processing-n:127.0.0.1:36183_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:36183_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:36183/collection1/ and leader is http://127.0.0.1:36183/collection1/
   [junit4]   2> 320065 INFO  (coreZkRegister-616-thread-1-processing-n:127.0.0.1:36183_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:36183_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:36183
   [junit4]   2> 320066 INFO  (coreZkRegister-616-thread-1-processing-n:127.0.0.1:36183_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:36183_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 320066 INFO  (coreZkRegister-616-thread-1-processing-n:127.0.0.1:36183_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:36183_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 320066 INFO  (coreZkRegister-616-thread-1-processing-n:127.0.0.1:36183_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:36183_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 320068 INFO  (zkCallback-118-thread-1-processing-n:127.0.0.1:36183_) [n:127.0.0.1:36183_    ] o.a.s.c.DistributedQueue NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 320068 INFO  (OverseerStateUpdate-94142656949518339-127.0.0.1:36183_-n_0000000000) [n:127.0.0.1:36183_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:36183",
   [junit4]   2>   "node_name":"127.0.0.1:36183_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"} current state version: 3
   [junit4]   2> 320069 INFO  (OverseerStateUpdate-94142656949518339-127.0.0.1:36183_-n_0000000000) [n:127.0.0.1:36183_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:36183",
   [junit4]   2>   "node_name":"127.0.0.1:36183_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 320131 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 320150 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 320171 INFO  (zkCallback-120-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@56a8d735 name:ZooKeeperConnection Watcher:127.0.0.1:46868/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 320172 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 320172 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 320172 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 320174 INFO  (zkCallback-118-thread-1-processing-n:127.0.0.1:36183_) [n:127.0.0.1:36183_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 320175 INFO  (zkCallback-120-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 0)
   [junit4]   2> 320177 INFO  (zkCallback-118-thread-1-processing-n:127.0.0.1:36183_) [n:127.0.0.1:36183_    ] o.a.s.c.c.ZkStateReader Updated cluster state version to 4
   [junit4]   2> 320178 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 320178 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with stateFormat=2
   [junit4]   2> 320178 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 320181 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 320193 INFO  (zkCallback-120-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated cluster state version to 4
   [junit4]   2> 320207 INFO  (zkCallback-121-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7193c81f name:ZooKeeperConnection Watcher:127.0.0.1:46868/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 320207 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 320207 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 320210 INFO  (zkCallback-118-thread-1-processing-n:127.0.0.1:36183_) [n:127.0.0.1:36183_    ] o.a.s.c.DistributedQueue NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 320212 INFO  (OverseerStateUpdate-94142656949518339-127.0.0.1:36183_-n_0000000000) [n:127.0.0.1:36183_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"create",
   [junit4]   2>   "name":"collection1",
   [junit4]   2>   "numShards":"4",
   [junit4]   2>   "stateFormat":"2"} current state version: 4
   [junit4]   2> 320212 INFO  (OverseerStateUpdate-94142656949518339-127.0.0.1:36183_-n_0000000000) [n:127.0.0.1:36183_    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 320315 INFO  (OverseerStateUpdate-94142656949518339-127.0.0.1:36183_-n_0000000000) [n:127.0.0.1:36183_    ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/collection1/state.json
   [junit4]   2> 320316 INFO  (zkCallback-120-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 320316 INFO  (zkCallback-118-thread-1-processing-n:127.0.0.1:36183_) [n:127.0.0.1:36183_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 320321 INFO  (zkCallback-120-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated cluster state version to 5
   [junit4]   2> 320322 INFO  (zkCallback-118-thread-1-processing-n:127.0.0.1:36183_) [n:127.0.0.1:36183_    ] o.a.s.c.c.ZkStateReader Updated cluster state version to 5
   [junit4]   2> 320881 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_D83020E37C7B824C-001/shard-1-001/cores/collection1
   [junit4]   2> 320883 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_D83020E37C7B824C-001/shard-1-001
   [junit4]   2> 320893 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [    ] o.e.j.s.Server jetty-9.2.11.v20150529
   [junit4]   2> 320985 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@26447417{/,null,AVAILABLE}
   [junit4]   2> 320986 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [    ] o.e.j.s.ServerConnector Started ServerConnector@32ab2823{HTTP/1.1}{127.0.0.1:39339}
   [junit4]   2> 320986 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [    ] o.e.j.s.Server Started @324430ms
   [junit4]   2> 320986 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=hdfs://localhost:40080/hdfs__localhost_40080__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-trunk_solr_build_solr-core_test_J0_temp_solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_D83020E37C7B824C-001_tempDir-002_jetty1, solrconfig=solrconfig.xml, hostContext=/, hostPort=39339, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_D83020E37C7B824C-001/shard-1-001/cores}
   [junit4]   2> 320987 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@c387f44
   [junit4]   2> 320987 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_D83020E37C7B824C-001/shard-1-001/'
   [junit4]   2> 321027 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_D83020E37C7B824C-001/shard-1-001/solr.xml
   [junit4]   2> 321050 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_D83020E37C7B824C-001/shard-1-001/cores
   [junit4]   2> 321050 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [    ] o.a.s.c.CoreContainer New CoreContainer 172389982
   [junit4]   2> 321051 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_D83020E37C7B824C-001/shard-1-001/]
   [junit4]   2> 321051 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_D83020E37C7B824C-001/shard-1-001/lib
   [junit4]   2> 321051 WARN  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_D83020E37C7B824C-001/shard-1-001/lib).
   [junit4]   2> 321074 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 321076 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 321076 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 321076 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 321080 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46868/solr
   [junit4]   2> 321080 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 321080 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 321100 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 321114 INFO  (zkCallback-123-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2a9f12d4 name:ZooKeeperConnection Watcher:127.0.0.1:46868 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 321114 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 321115 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 321141 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [n:127.0.0.1:39339_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 321198 INFO  (zkCallback-124-thread-1-processing-n:127.0.0.1:39339_) [n:127.0.0.1:39339_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@21acce16 name:ZooKeeperConnection Watcher:127.0.0.1:46868/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 321198 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [n:127.0.0.1:39339_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 321208 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [n:127.0.0.1:39339_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 322214 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [n:127.0.0.1:39339_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:39339_
   [junit4]   2> 322214 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [n:127.0.0.1:39339_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:39339_
   [junit4]   2> 322217 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [n:127.0.0.1:39339_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 322218 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [n:127.0.0.1:39339_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94142656949518343-127.0.0.1:39339_-n_0000000001
   [junit4]   2> 322218 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [n:127.0.0.1:39339_    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/94142656949518339-127.0.0.1:36183_-n_0000000000 to know if I could be the leader
   [junit4]   2> 322219 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [n:127.0.0.1:39339_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 322219 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [n:127.0.0.1:39339_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 322220 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [n:127.0.0.1:39339_    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_D83020E37C7B824C-001/shard-1-001/cores
   [junit4]   2> 322221 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [n:127.0.0.1:39339_    ] o.a.s.c.SolrCore Created CoreDescriptor: {name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, loadOnStartup=true, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_D83020E37C7B824C-001/shard-1-001/cores/collection1, collection=collection1, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_D83020E37C7B824C-001/shard-1-001/cores/collection1/, coreNodeName=, dataDir=data/, shard=}
   [junit4]   2> 322221 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [n:127.0.0.1:39339_    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_D83020E37C7B824C-001/shard-1-001/cores/collection1/
   [junit4]   2> 322221 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D83020E37C7B824C]) [n:127.0.0.1:39339_    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 322238 INFO  (coreLoadExecutor-633-thread-1-processing-n:127.0.0.1:39339_) [n:127.0.0.1:39339_ c:collection1   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 322239 INFO  (coreLoadExecutor-633-thread-1-processing-n:127.0.0.1:39339_) [n:127.0.0.1:39339_ c:collection1   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 322240 INFO  (zkCallback-118-thread-1-processing-n:127.0.0.1:36183_) [n:127.0.0.1:36183_    ] o.a.s.c.DistributedQueue NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 322240 INFO  (coreLoadExecutor-633-thread-1-processing-n:127.0.0.1:39339_) [n:127.0.0.1:39339_ c:collection1   x:collection1] o.a.s.c.ZkController Registering watch for external collection collection1
   [junit4]   2> 322240 INFO  (coreLoadExecutor-633-thread-1-processing-n:127.0.0.1:39339_) [n:127.0.0.1:39339_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader addZkWatch collection1
   [junit4]   2> 322240 INFO  (OverseerStateUpdate-94142656949518339-127.0.0.1:36183_-n_0000000000) [n:127.0.0.1:36183_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:39339",
   [junit4]   2>   "node_name":"127.0.0.1:39339_",
   [junit4]   2>   "numShards":"4",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "operation":"state"} current state version: 5
   [junit4]   2> 322241 INFO  (coreLoadExecutor-633-thread-1-processing-n:127.0.0.1:39339_) [n:127.0.0.1:39339_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Updating collection state at /collections/collection1/state.json from ZooKeeper... 
   [junit4]   2> 322241 INFO  (OverseerStateUpdate-94142656949518339-127.0.0.1:36183_-n_0000000000) [n:127.0.0.1:36183_    ] o.a.s.c.o.ReplicaMutator Update state numShards=4 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:39339",
   [junit4]   2>   "node_name":"127.0.0.1:39339_",
   [junit4]   2>   "numShards":"4",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 322241 INFO  (OverseerStateUpdate-94142656949518339-127.0.0.1:36183_-n_0000000000) [n:127.0.0.1:36183_    ] o.a.s.c.o.ReplicaMutator Collection already exists with numShards=4
   [junit4]   2> 322241 INFO  (OverseerStateUpdate-94142656949518339-127.0.0.1:36183_-n_0000000000) [n:127.0.0.1:36183_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2
   [junit4]   2> 322242 INFO  (coreLoadExecutor-633-thread-1-processing-n:127.0.0.1:39339_) [n:127.0.0.1:39339_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Updating data for collection1 to ver 0 
   [junit4]   2> 322242 INFO  (coreLoadExecutor-633-thread-1-processing-n:127.0.0.1:39339_) [n:127.0.0.1:39339_ c:collection1   x:collection1] o.a.s.c.ZkController look for our core node nam

[...truncated too long message...]

wn(MiniDFSCluster.java:1699)
   [junit4]   2> 	at org.apache.solr.cloud.hdfs.HdfsTestUtil.teardownClass(HdfsTestUtil.java:197)
   [junit4]   2> 	at org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.teardownClass(HdfsCollectionsAPIDistributedZkTest.java:50)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   [junit4]   2> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2> 	at java.lang.reflect.Method.invoke(Method.java:483)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:799)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> Caused by: java.lang.NullPointerException
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.blockmanagement.BlocksMap.size(BlocksMap.java:198)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.getTotalBlocks(BlockManager.java:3291)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlocksTotal(FSNamesystem.java:6223)
   [junit4]   2> 	... 53 more
   [junit4]   2> 3639267 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[D83020E37C7B824C]-worker) [n:127.0.0.1:34457_zw c:addReplicaColl s:shard1 r:core_node7 x:propertyDotName] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> Jul 10, 2015 1:31:07 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 2 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene53): {other_tl1=PostingsFormat(name=Memory doPackFST= true), range_facet_l_dv=BlockTreeOrds(blocksize=128), _version_=PostingsFormat(name=Memory doPackFST= true), multiDefault=PostingsFormat(name=Direct), a_t=PostingsFormat(name=Direct), intDefault=PostingsFormat(name=Memory doPackFST= true), a_i1=PostingsFormat(name=Direct), id=BlockTreeOrds(blocksize=128), range_facet_i_dv=PostingsFormat(name=Direct), text=PostingsFormat(name=LuceneVarGapFixedInterval), range_facet_l=PostingsFormat(name=Direct), timestamp=PostingsFormat(name=Direct)}, docValues:{range_facet_l_dv=DocValuesFormat(name=Asserting), range_facet_i_dv=DocValuesFormat(name=Memory), timestamp=DocValuesFormat(name=Memory)}, sim=DefaultSimilarity, locale=mk_MK, timezone=Pacific/Fakaofo
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.8.0_25 (64-bit)/cpus=4,threads=2,free=171640704,total=525860864
   [junit4]   2> NOTE: All tests run in this JVM: [SuggestComponentTest, RankQueryTest, DateRangeFieldTest, DeleteLastCustomShardedReplicaTest, TestManagedSchema, TestOverriddenPrefixQueryForCustomFieldType, CacheHeaderTest, DisMaxRequestHandlerTest, TestDownShardTolerantSearch, TestRandomRequestDistribution, TestReRankQParserPlugin, TestManagedSchemaDynamicFieldResource, TestSolrConfigHandlerCloud, JsonLoaderTest, CleanupOldIndexTest, SpatialHeatmapFacetsTest, TestDistributedMissingSort, TestSolrDeletionPolicy2, ShowFileRequestHandlerTest, TestDefaultStatsCache, ClusterStateUpdateTest, DirectSolrConnectionTest, SpellingQueryConverterTest, OpenCloseCoreStressTest, TestSolrCoreProperties, TestWriterPerf, BadCopyFieldTest, ZkControllerTest, SpatialFilterTest, TestJettySolrRunner, TestConfigSets, HdfsRecoverLeaseTest, TestCustomDocTransformer, RulesTest, HdfsBasicDistributedZkTest, TestReplicaProperties, DistanceFunctionTest, ConvertedLegacyTest, TestSchemaVersionResource, ChaosMonkeySafeLeaderTest, DistributedQueueTest, TestCoreDiscovery, TestLRUCache, ZkCLITest, DocExpirationUpdateProcessorFactoryTest, CollectionsAPIDistributedZkTest, TestSerializedLuceneMatchVersion, FieldAnalysisRequestHandlerTest, StatelessScriptUpdateProcessorFactoryTest, TestAtomicUpdateErrorCases, AddSchemaFieldsUpdateProcessorFactoryTest, SolrCloudExampleTest, ResourceLoaderTest, SolrXmlInZkTest, DocumentBuilderTest, FieldMutatingUpdateProcessorTest, TestSQLHandler, CoreAdminRequestStatusTest, SolrPluginUtilsTest, PreAnalyzedUpdateProcessorTest, QueryResultKeyTest, TestRTGBase, SpellPossibilityIteratorTest, TestLMDirichletSimilarityFactory, TestFastWriter, TestStressVersions, SolrCmdDistributorTest, TestHashPartitioner, AnalysisErrorHandlingTest, InfoHandlerTest, TestPHPSerializedResponseWriter, TestSchemaNameResource, CoreAdminHandlerTest, TestClusterStateMutator, TestLFUCache, ScriptEngineTest, TestStandardQParsers, AsyncCallRequestStatusResponseTest, WordBreakSolrSpellCheckerTest, TestFoldingMultitermQuery, TestFiltering, AnalysisAfterCoreReloadTest, TestElisionMultitermQuery, TestReload, TestCoreContainer, StatsComponentTest, TestFunctionQuery, BasicFunctionalityTest, TestLazyCores, TestIndexSearcher, SimplePostToolTest, SignatureUpdateProcessorFactoryTest, SpellCheckCollatorTest, SuggesterTSTTest, SuggesterWFSTTest, TestPseudoReturnFields, TestWordDelimiterFilterFactory, TestRemoteStreaming, TestSolrDeletionPolicy1, XsltUpdateRequestHandlerTest, LukeRequestHandlerTest, PrimitiveFieldTypeTest, FileBasedSpellCheckerTest, RequestHandlersTest, RequiredFieldsTest, FastVectorHighlighterTest, TestCollationField, UpdateRequestProcessorFactoryTest, JSONWriterTest, QueryParsingTest, PingRequestHandlerTest, TestComponentsName, TestBinaryResponseWriter, SOLR749Test, UpdateParamsTest, TestQuerySenderNoQuery, ResponseLogComponentTest, BadComponentTest, TestMergePolicyConfig, TestDocSet, TestSearchPerf, TestFuzzyAnalyzedSuggestions, TestPhraseSuggestions, TestCharFilters, EchoParamsTest, TestPerFieldSimilarity, TestBM25SimilarityFactory, TestJmxMonitoredMap, PluginInfoTest, PreAnalyzedFieldTest, TestSolrJ, TestDocumentBuilder, SliceStateTest, SystemInfoHandlerTest, CursorPagingTest, TestTolerantSearch, AliasIntegrationTest, BaseCdcrDistributedZkTest, CloudExitableDirectoryReaderTest, ConcurrentDeleteAndCreateCollectionTest, DistribCursorPagingTest, LeaderFailoverAfterPartitionTest, MigrateRouteKeyTest, OutOfBoxZkACLAndCredentialsProvidersTest, OverriddenZkACLAndCredentialsProvidersTest, OverseerStatusTest, RemoteQueryErrorTest, SSLMigrationTest, SaslZkACLProviderTest, ShardSplitTest, TestMiniSolrCloudCluster, TestRequestStatusCollectionAPI, TestShortCircuitedRequests, VMParamsZkACLAndCredentialsProvidersTest, HdfsCollectionsAPIDistributedZkTest]
   [junit4] Completed [457/514] on J1 in 709.25s, 1 test, 1 error <<< FAILURES!

[...truncated 189 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:533: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:474: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:61: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/extra-targets.xml:39: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build.xml:230: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/common-build.xml:513: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/common-build.xml:1426: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/common-build.xml:981: There were test failures: 514 suites, 2022 tests, 1 error, 1 failure, 20 ignored (10 assumptions)

Total time: 168 minutes 50 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
No prior successful build to compare, so performing full copy of artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-NightlyTests-trunk - Build # 749 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-trunk/749/

1 tests failed.
FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=6560, name=collection5, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=6560, name=collection5, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:39297/yc_suz/mw: Could not find collection : awholynewstresscollection_collection5_1
	at __randomizedtesting.SeedInfo.seed([F6FA78EB9F8891F]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:560)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:234)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:226)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:376)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:328)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1086)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:856)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:799)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:894)




Build Log:
[...truncated 10488 lines...]
   [junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_F6FA78EB9F8891F-001/init-core-data-001
   [junit4]   2> 1279437 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[F6FA78EB9F8891F]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (false)
   [junit4]   2> 1279439 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[F6FA78EB9F8891F]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /yc_suz/mw
   [junit4]   2> 1279441 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 1279442 INFO  (Thread-5036) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1279442 INFO  (Thread-5036) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 1279542 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [    ] o.a.s.c.ZkTestServer start zk server on port:43713
   [junit4]   2> 1279542 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1279543 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1279547 INFO  (zkCallback-193-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@448643b5 name:ZooKeeperConnection Watcher:127.0.0.1:43713 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1279547 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1279548 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1279548 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 1279551 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1279552 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1279557 INFO  (zkCallback-194-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@637ab6fc name:ZooKeeperConnection Watcher:127.0.0.1:43713/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1279559 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1279559 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1279559 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 1279564 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 1279566 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 1279567 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 1279568 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 1279568 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 1279571 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 1279571 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 1279573 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1279573 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1279575 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 1279575 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 1279576 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 1279577 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 1279578 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 1279578 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 1279580 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 1279580 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 1279582 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1279582 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1279583 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1279584 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1279585 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 1279585 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 1279587 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 1279587 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 1279590 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1279590 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1279592 INFO  (zkCallback-195-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2580d1c2 name:ZooKeeperConnection Watcher:127.0.0.1:43713/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1279592 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1279592 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1279887 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_F6FA78EB9F8891F-001/control-001/cores/collection1
   [junit4]   2> 1279907 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [    ] o.e.j.s.Server jetty-9.2.11.v20150529
   [junit4]   2> 1279909 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@66ebc1dd{/yc_suz/mw,null,AVAILABLE}
   [junit4]   2> 1280092 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [    ] o.e.j.s.ServerConnector Started ServerConnector@40128c8f{SSL-http/1.1}{127.0.0.1:55380}
   [junit4]   2> 1280092 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [    ] o.e.j.s.Server Started @1283667ms
   [junit4]   2> 1280092 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_F6FA78EB9F8891F-001/tempDir-001/control/data, hostContext=/yc_suz/mw, hostPort=55380, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_F6FA78EB9F8891F-001/control-001/cores}
   [junit4]   2> 1280102 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@4e0e2f2a
   [junit4]   2> 1280102 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_F6FA78EB9F8891F-001/control-001/'
   [junit4]   2> 1280119 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1280127 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1280129 INFO  (zkCallback-196-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@69e476fe name:ZooKeeperConnection Watcher:127.0.0.1:43713/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1280129 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1280130 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1280131 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1280131 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_F6FA78EB9F8891F-001/control-001/solr.xml
   [junit4]   2> 1280137 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_F6FA78EB9F8891F-001/control-001/cores
   [junit4]   2> 1280138 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [    ] o.a.s.c.CoreContainer New CoreContainer 176611399
   [junit4]   2> 1280138 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_F6FA78EB9F8891F-001/control-001/]
   [junit4]   2> 1280138 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_F6FA78EB9F8891F-001/control-001/lib
   [junit4]   2> 1280138 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_F6FA78EB9F8891F-001/control-001/lib).
   [junit4]   2> 1280147 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1280151 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1280153 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1280153 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1280154 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:43713/solr
   [junit4]   2> 1280154 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1280154 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1280154 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1280156 INFO  (zkCallback-198-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@214722ea name:ZooKeeperConnection Watcher:127.0.0.1:43713 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1280157 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1280157 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1280167 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [n:127.0.0.1:55380_yc_suz%2Fmw    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1280168 INFO  (zkCallback-199-thread-1-processing-n:127.0.0.1:55380_yc_suz%2Fmw) [n:127.0.0.1:55380_yc_suz%2Fmw    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3b277775 name:ZooKeeperConnection Watcher:127.0.0.1:43713/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1280169 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [n:127.0.0.1:55380_yc_suz%2Fmw    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1280170 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [n:127.0.0.1:55380_yc_suz%2Fmw    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 1280172 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [n:127.0.0.1:55380_yc_suz%2Fmw    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 1280174 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [n:127.0.0.1:55380_yc_suz%2Fmw    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 1280176 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [n:127.0.0.1:55380_yc_suz%2Fmw    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 1280177 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [n:127.0.0.1:55380_yc_suz%2Fmw    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 1280180 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [n:127.0.0.1:55380_yc_suz%2Fmw    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 1280181 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [n:127.0.0.1:55380_yc_suz%2Fmw    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 1280182 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [n:127.0.0.1:55380_yc_suz%2Fmw    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 1280183 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [n:127.0.0.1:55380_yc_suz%2Fmw    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:55380_yc_suz%2Fmw
   [junit4]   2> 1280184 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [n:127.0.0.1:55380_yc_suz%2Fmw    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:55380_yc_suz%2Fmw
   [junit4]   2> 1280185 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [n:127.0.0.1:55380_yc_suz%2Fmw    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 1280186 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [n:127.0.0.1:55380_yc_suz%2Fmw    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 1280188 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [n:127.0.0.1:55380_yc_suz%2Fmw    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1280188 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [n:127.0.0.1:55380_yc_suz%2Fmw    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94245069702037509-127.0.0.1:55380_yc_suz%2Fmw-n_0000000000
   [junit4]   2> 1280189 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [n:127.0.0.1:55380_yc_suz%2Fmw    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:55380_yc_suz%2Fmw
   [junit4]   2> 1280189 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [n:127.0.0.1:55380_yc_suz%2Fmw    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 1280196 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [n:127.0.0.1:55380_yc_suz%2Fmw    ] o.a.s.c.Overseer Overseer (id=94245069702037509-127.0.0.1:55380_yc_suz%2Fmw-n_0000000000) starting
   [junit4]   2> 1280198 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [n:127.0.0.1:55380_yc_suz%2Fmw    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 1280203 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [n:127.0.0.1:55380_yc_suz%2Fmw    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 1280204 INFO  (OverseerStateUpdate-94245069702037509-127.0.0.1:55380_yc_suz%2Fmw-n_0000000000) [n:127.0.0.1:55380_yc_suz%2Fmw    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 1280207 INFO  (OverseerCollectionProcessor-94245069702037509-127.0.0.1:55380_yc_suz%2Fmw-n_0000000000) [n:127.0.0.1:55380_yc_suz%2Fmw    ] o.a.s.c.OverseerCollectionProcessor Process current queue of collection creations
   [junit4]   2> 1280211 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [n:127.0.0.1:55380_yc_suz%2Fmw    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1280214 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [n:127.0.0.1:55380_yc_suz%2Fmw    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1280215 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [n:127.0.0.1:55380_yc_suz%2Fmw    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1280215 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [n:127.0.0.1:55380_yc_suz%2Fmw    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_F6FA78EB9F8891F-001/control-001/cores
   [junit4]   2> 1280216 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [n:127.0.0.1:55380_yc_suz%2Fmw    ] o.a.s.c.SolrCore Created CoreDescriptor: {name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, loadOnStartup=true, configSetProperties=configsetprops.json, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_F6FA78EB9F8891F-001/control-001/cores/collection1, collection=control_collection, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_F6FA78EB9F8891F-001/control-001/cores/collection1/, coreNodeName=, dataDir=data/, shard=}
   [junit4]   2> 1280217 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [n:127.0.0.1:55380_yc_suz%2Fmw    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_F6FA78EB9F8891F-001/control-001/cores/collection1/
   [junit4]   2> 1280217 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [n:127.0.0.1:55380_yc_suz%2Fmw    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 1280217 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [n:127.0.0.1:55380_yc_suz%2Fmw    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1
   [junit4]   2> 1280218 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [n:127.0.0.1:55380_yc_suz%2Fmw    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1280218 INFO  (coreLoadExecutor-933-thread-1-processing-n:127.0.0.1:55380_yc_suz%2Fmw) [n:127.0.0.1:55380_yc_suz%2Fmw c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 1280218 INFO  (coreLoadExecutor-933-thread-1-processing-n:127.0.0.1:55380_yc_suz%2Fmw) [n:127.0.0.1:55380_yc_suz%2Fmw c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1280219 INFO  (zkCallback-199-thread-1-processing-n:127.0.0.1:55380_yc_suz%2Fmw) [n:127.0.0.1:55380_yc_suz%2Fmw    ] o.a.s.c.DistributedQueue NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 1280219 INFO  (coreLoadExecutor-933-thread-1-processing-n:127.0.0.1:55380_yc_suz%2Fmw) [n:127.0.0.1:55380_yc_suz%2Fmw c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 1280220 INFO  (OverseerStateUpdate-94245069702037509-127.0.0.1:55380_yc_suz%2Fmw-n_0000000000) [n:127.0.0.1:55380_yc_suz%2Fmw    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:55380/yc_suz/mw",
   [junit4]   2>   "node_name":"127.0.0.1:55380_yc_suz%2Fmw",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 1280220 INFO  (OverseerStateUpdate-94245069702037509-127.0.0.1:55380_yc_suz%2Fmw-n_0000000000) [n:127.0.0.1:55380_yc_suz%2Fmw    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:55380/yc_suz/mw",
   [junit4]   2>   "node_name":"127.0.0.1:55380_yc_suz%2Fmw",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 1280221 INFO  (OverseerStateUpdate-94245069702037509-127.0.0.1:55380_yc_suz%2Fmw-n_0000000000) [n:127.0.0.1:55380_yc_suz%2Fmw    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 1280221 INFO  (OverseerStateUpdate-94245069702037509-127.0.0.1:55380_yc_suz%2Fmw-n_0000000000) [n:127.0.0.1:55380_yc_suz%2Fmw    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 1280222 INFO  (zkCallback-199-thread-1-processing-n:127.0.0.1:55380_yc_suz%2Fmw) [n:127.0.0.1:55380_yc_suz%2Fmw    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1280224 INFO  (zkCallback-199-thread-1-processing-n:127.0.0.1:55380_yc_suz%2Fmw) [n:127.0.0.1:55380_yc_suz%2Fmw    ] o.a.s.c.c.ZkStateReader Updated cluster state version to 1
   [junit4]   2> 1281219 INFO  (coreLoadExecutor-933-thread-1-processing-n:127.0.0.1:55380_yc_suz%2Fmw) [n:127.0.0.1:55380_yc_suz%2Fmw c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1281219 INFO  (coreLoadExecutor-933-thread-1-processing-n:127.0.0.1:55380_yc_suz%2Fmw) [n:127.0.0.1:55380_yc_suz%2Fmw c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 1281220 INFO  (coreLoadExecutor-933-thread-1-processing-n:127.0.0.1:55380_yc_suz%2Fmw) [n:127.0.0.1:55380_yc_suz%2Fmw c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 1281220 INFO  (coreLoadExecutor-933-thread-1-processing-n:127.0.0.1:55380_yc_suz%2Fmw) [n:127.0.0.1:55380_yc_suz%2Fmw c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 1281221 INFO  (coreLoadExecutor-933-thread-1-processing-n:127.0.0.1:55380_yc_suz%2Fmw) [n:127.0.0.1:55380_yc_suz%2Fmw c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1281221 INFO  (coreLoadExecutor-933-thread-1-processing-n:127.0.0.1:55380_yc_suz%2Fmw) [n:127.0.0.1:55380_yc_suz%2Fmw c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_F6FA78EB9F8891F-001/control-001/cores/collection1/'
   [junit4]   2> 1281235 INFO  (coreLoadExecutor-933-thread-1-processing-n:127.0.0.1:55380_yc_suz%2Fmw) [n:127.0.0.1:55380_yc_suz%2Fmw c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 1281239 INFO  (coreLoadExecutor-933-thread-1-processing-n:127.0.0.1:55380_yc_suz%2Fmw) [n:127.0.0.1:55380_yc_suz%2Fmw c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 1281243 INFO  (coreLoadExecutor-933-thread-1-processing-n:127.0.0.1:55380_yc_suz%2Fmw) [n:127.0.0.1:55380_yc_suz%2Fmw c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.0.0
   [junit4]   2> 1281251 INFO  (coreLoadExecutor-933-thread-1-processing-n:127.0.0.1:55380_yc_suz%2Fmw) [n:127.0.0.1:55380_yc_suz%2Fmw c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1281252 INFO  (coreLoadExecutor-933-thread-1-processing-n:127.0.0.1:55380_yc_suz%2Fmw) [n:127.0.0.1:55380_yc_suz%2Fmw c:control_collection   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 1281257 INFO  (coreLoadExecutor-933-thread-1-processing-n:127.0.0.1:55380_yc_suz%2Fmw) [n:127.0.0.1:55380_yc_suz%2Fmw c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1281352 INFO  (coreLoadExecutor-933-thread-1-processing-n:127.0.0.1:55380_yc_suz%2Fmw) [n:127.0.0.1:55380_yc_suz%2Fmw c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 1281358 INFO  (coreLoadExecutor-933-thread-1-processing-n:127.0.0.1:55380_yc_suz%2Fmw) [n:127.0.0.1:55380_yc_suz%2Fmw c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 1281359 INFO  (coreLoadExecutor-933-thread-1-processing-n:127.0.0.1:55380_yc_suz%2Fmw) [n:127.0.0.1:55380_yc_suz%2Fmw c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 1281367 INFO  (coreLoadExecutor-933-thread-1-processing-n:127.0.0.1:55380_yc_suz%2Fmw) [n:127.0.0.1:55380_yc_suz%2Fmw c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1281370 INFO  (coreLoadExecutor-933-thread-1-processing-n:127.0.0.1:55380_yc_suz%2Fmw) [n:127.0.0.1:55380_yc_suz%2Fmw c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1281372 INFO  (coreLoadExecutor-933-thread-1-processing-n:127.0.0.1:55380_yc_suz%2Fmw) [n:127.0.0.1:55380_yc_suz%2Fmw c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1281373 WARN  (coreLoadExecutor-933-thread-1-processing-n:127.0.0.1:55380_yc_suz%2Fmw) [n:127.0.0.1:55380_yc_suz%2Fmw c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 1281373 WARN  (coreLoadExecutor-933-thread-1-processing-n:127.0.0.1:55380_yc_suz%2Fmw) [n:127.0.0.1:55380_yc_suz%2Fmw c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 1281373 INFO  (coreLoadExecutor-933-thread-1-processing-n:127.0.0.1:55380_yc_suz%2Fmw) [n:127.0.0.1:55380_yc_suz%2Fmw c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1281374 WARN  (coreLoadExecutor-933-thread-1-processing-n:127.0.0.1:55380_yc_suz%2Fmw) [n:127.0.0.1:55380_yc_suz%2Fmw c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 1281374 WARN  (coreLoadExecutor-933-thread-1-processing-n:127.0.0.1:55380_yc_suz%2Fmw) [n:127.0.0.1:55380_yc_suz%2Fmw c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 1281375 INFO  (coreLoadExecutor-933-thread-1-processing-n:127.0.0.1:55380_yc_suz%2Fmw) [n:127.0.0.1:55380_yc_suz%2Fmw c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1
   [junit4]   2> 1281375 INFO  (coreLoadExecutor-933-thread-1-processing-n:127.0.0.1:55380_yc_suz%2Fmw) [n:127.0.0.1:55380_yc_suz%2Fmw c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 1281375 INFO  (coreLoadExecutor-933-thread-1-processing-n:127.0.0.1:55380_yc_suz%2Fmw) [n:127.0.0.1:55380_yc_suz%2Fmw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 1281375 INFO  (coreLoadExecutor-933-thread-1-processing-n:127.0.0.1:55380_yc_suz%2Fmw) [n:127.0.0.1:55380_yc_suz%2Fmw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_F6FA78EB9F8891F-001/control-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 1281375 INFO  (coreLoadExecutor-933-thread-1-processing-n:127.0.0.1:55380_yc_suz%2Fmw) [n:127.0.0.1:55380_yc_suz%2Fmw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@437f9f40
   [junit4]   2> 1281376 INFO  (coreLoadExecutor-933-thread-1-processing-n:127.0.0.1:55380_yc_suz%2Fmw) [n:127.0.0.1:55380_yc_suz%2Fmw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_F6FA78EB9F8891F-001/control-001/cores/collection1/data
   [junit4]   2> 1281376 INFO  (coreLoadExecutor-933-thread-1-processing-n:127.0.0.1:55380_yc_suz%2Fmw) [n:127.0.0.1:55380_yc_suz%2Fmw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_F6FA78EB9F8891F-001/control-001/cores/collection1/data/index/
   [junit4]   2> 1281376 WARN  (coreLoadExecutor-933-thread-1-processing-n:127.0.0.1:55380_yc_suz%2Fmw) [n:127.0.0.1:55380_yc_suz%2Fmw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_F6FA78EB9F8891F-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 1281377 INFO  (coreLoadExecutor-933-thread-1-processing-n:127.0.0.1:55380_yc_suz%2Fmw) [n:127.0.0.1:55380_yc_suz%2Fmw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_F6FA78EB9F8891F-001/control-001/cores/collection1/data/index
   [junit4]   2> 1281377 INFO  (coreLoadExecutor-933-thread-1-processing-n:127.0.0.1:55380_yc_suz%2Fmw) [n:127.0.0.1:55380_yc_suz%2Fmw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@ccf7932
   [junit4]   2> 1281377 INFO  (coreLoadExecutor-933-thread-1-processing-n:127.0.0.1:55380_yc_suz%2Fmw) [n:127.0.0.1:55380_yc_suz%2Fmw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@6a9aef3e lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5484fd30),segFN=segments_1,generation=1}
   [junit4]   2> 1281378 INFO  (coreLoadExecutor-933-thread-1-processing-n:127.0.0.1:55380_yc_suz%2Fmw) [n:127.0.0.1:55380_yc_suz%2Fmw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 1281378 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:55380_yc_suz%2Fmw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_F6FA78EB9F8891F-001/control-001/cores/collection1/data/
   [junit4]   2> 1281378 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:55380_yc_suz%2Fmw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_F6FA78EB9F8891F-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 1281386 INFO  (coreLoadExecutor-933-thread-1-processing-n:127.0.0.1:55380_yc_suz%2Fmw) [n:127.0.0.1:55380_yc_suz%2Fmw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 1281386 INFO  (coreLoadExecutor-933-thread-1-processing-n:127.0.0.1:55380_yc_suz%2Fmw) [n:127.0.0.1:55380_yc_suz%2Fmw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 1281387 INFO  (coreLoadExecutor-933-thread-1-processing-n:127.0.0.1:55380_yc_suz%2Fmw) [n:127.0.0.1:55380_yc_suz%2Fmw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 1281387 INFO  (coreLoadExecutor-933-thread-1-processing-n:127.0.0.1:55380_yc_suz%2Fmw) [n:127.0.0.1:55380_yc_suz%2Fmw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1281387 INFO  (coreLoadExecutor-933-thread-1-processing-n:127.0.0.1:55380_yc_suz%2Fmw) [n:127.0.0.1:55380_yc_suz%2Fmw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1281387 INFO  (coreLoadExecutor-933-thread-1-processing-n:127.0.0.1:55380_yc_suz%2Fmw) [n:127.0.0.1:55380_yc_suz%2Fmw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 1281388 INFO  (coreLoadExecutor-933-thread-1-processing-n:127.0.0.1:55380_yc_suz%2Fmw) [n:127.0.0.1:55380_yc_suz%2Fmw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1281388 INFO  (coreLoadExecutor-933-thread-1-processing-n:127.0.0.1:55380_yc_suz%2Fmw) [n:127.0.0.1:55380_yc_suz%2Fmw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1281388 INFO  (coreLoadExecutor-933-thread-1-processing-n:127.0.0.1:55380_yc_suz%2Fmw) [n:127.0.0.1:55380_yc_suz%2Fmw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 1281390 INFO  (coreLoadExecutor-933-thread-1-processing-n:127.0.0.1:55380_yc_suz%2Fmw) [n:127.0.0.1:55380_yc_suz%2Fmw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1281391 INFO  (coreLoadExecutor-933-thread-1-processing-n:127.0.0.1:55380_yc_suz%2Fmw) [n:127.0.0.1:55380_yc_suz%2Fmw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1281392 INFO  (coreLoadExecutor-933-thread-1-processing-n:127.0.0.1:55380_yc_suz%2Fmw) [n:127.0.0.1:55380_yc_suz%2Fmw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1281393 INFO  (coreLoadExecutor-933-thread-1-processing-n:127.0.0.1:55380_yc_suz%2Fmw) [n:127.0.0.1:55380_yc_suz%2Fmw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1281397 INFO  (coreLoadExecutor-933-thread-1-processing-n:127.0.0.1:55380_yc_suz%2Fmw) [n:127.0.0.1:55380_yc_suz%2Fmw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/mbeans,standard,/update/csv,/update/json/docs,/admin/luke,/admin/segments,/get,/admin/system,/replication,/admin/properties,/config,/schema,/admin/plugins,/admin/logging,/update/json,/admin/threads,/admin/ping,/update,/admin/file
   [junit4]   2> 1281397 INFO  (coreLoadExecutor-933-thread-1-processing-n:127.0.0.1:55380_yc_suz%2Fmw) [n:127.0.0.1:55380_yc_suz%2Fmw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 1281398 INFO  (coreLoadExecutor-933-thread-1-processing-n:127.0.0.1:55380_yc_suz%2Fmw) [n:127.0.0.1:55380_yc_suz%2Fmw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1281398 INFO  (coreLoadExecutor-933-thread-1-processing-n:127.0.0.1:55380_yc_suz%2Fmw) [n:127.0.0.1:55380_yc_suz%2Fmw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1281399 INFO  (coreLoadExecutor-933-thread-1-processing-n:127.0.0.1:55380_yc_suz%2Fmw) [n:127.0.0.1:55380_yc_suz%2Fmw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 1281399 INFO  (coreLoadExecutor-933-thread-1-processing-n:127.0.0.1:55380_yc_suz%2Fmw) [n:127.0.0.1:55380_yc_suz%2Fmw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 1281400 INFO  (coreLoadExecutor-933-thread-1-processing-n:127.0.0.1:55380_yc_suz%2Fmw) [n:127.0.0.1:55380_yc_suz%2Fmw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=15, maxMergeAtOnceExplicit=23, maxMergedSegmentMB=60.7734375, floorSegmentMB=1.296875, forceMergeDeletesPctAllowed=18.142522495386093, segmentsPerTier=23.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
   [junit4]   2> 1281400 INFO  (coreLoadExecutor-933-thread-1-processing-n:127.0.0.1:55380_yc_suz%2Fmw) [n:127.0.0.1:55380_yc_suz%2Fmw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@6a9aef3e lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5484fd30),segFN=segments_1,generation=1}
   [junit4]   2> 1281400 INFO  (coreLoadExecutor-933-thread-1-processing-n:127.0.0.1:55380_yc_suz%2Fmw) [n:127.0.0.1:55380_yc_suz%2Fmw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 1281400 INFO  (coreLoadExecutor-933-thread-1-processing-n:127.0.0.1:55380_yc_suz%2Fmw) [n:127.0.0.1:55380_yc_suz%2Fmw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@6e9e5746[collection1] main
   [junit4]   2> 1281400 INFO  (coreLoadExecutor-933-thread-1-processing-n:127.0.0.1:55380_yc_suz%2Fmw) [n:127.0.0.1:55380_yc_suz%2Fmw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 1281401 INFO  (coreLoadExecutor-933-thread-1-processing-n:127.0.0.1:55380_yc_suz%2Fmw) [n:127.0.0.1:55380_yc_suz%2Fmw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1281401 INFO  (coreLoadExecutor-933-thread-1-processing-n:127.0.0.1:55380_yc_suz%2Fmw) [n:127.0.0.1:55380_yc_suz%2Fmw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 1281402 INFO  (coreLoadExecutor-933-thread-1-processing-n:127.0.0.1:55380_yc_suz%2Fmw) [n:127.0.0.1:55380_yc_suz%2Fmw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1281402 INFO  (coreLoadExecutor-933-thread-1-processing-n:127.0.0.1:55380_yc_suz%2Fmw) [n:127.0.0.1:55380_yc_suz%2Fmw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 1281402 INFO  (coreLoadExecutor-933-thread-1-processing-n:127.0.0.1:55380_yc_suz%2Fmw) [n:127.0.0.1:55380_yc_suz%2Fmw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1281402 INFO  (coreLoadExecutor-933-thread-1-processing-n:127.0.0.1:55380_yc_suz%2Fmw) [n:127.0.0.1:55380_yc_suz%2Fmw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 1281403 INFO  (coreLoadExecutor-933-thread-1-processing-n:127.0.0.1:55380_yc_suz%2Fmw) [n:127.0.0.1:55380_yc_suz%2Fmw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1281403 INFO  (coreLoadExecutor-933-thread-1-processing-n:127.0.0.1:55380_yc_suz%2Fmw) [n:127.0.0.1:55380_yc_suz%2Fmw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 1281403 INFO  (coreLoadExecutor-933-thread-1-processing-n:127.0.0.1:55380_yc_suz%2Fmw) [n:127.0.0.1:55380_yc_suz%2Fmw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1281403 INFO  (coreLoadExecutor-933-thread-1-processing-n:127.0.0.1:55380_yc_suz%2Fmw) [n:127.0.0.1:55380_yc_suz%2Fmw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 1281403 INFO  (coreLoadExecutor-933-thread-1-processing-n:127.0.0.1:55380_yc_suz%2Fmw) [n:127.0.0.1:55380_yc_suz%2Fmw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 1281404 INFO  (coreLoadExecutor-933-thread-1-processing-n:127.0.0.1:55380_yc_suz%2Fmw) [n:127.0.0.1:55380_yc_suz%2Fmw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 1281404 INFO  (coreLoadExecutor-933-thread-1-processing-n:127.0.0.1:55380_yc_suz%2Fmw) [n:127.0.0.1:55380_yc_suz%2Fmw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1507921117288857600
   [junit4]   2> 1281404 INFO  (searcherExecutor-934-thread-1-processing-n:127.0.0.1:55380_yc_suz%2Fmw x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:55380_yc_suz%2Fmw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@6e9e5746[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1281406 INFO  (coreLoadExecutor-933-thread-1-processing-n:127.0.0.1:55380_yc_suz%2Fmw) [n:127.0.0.1:55380_yc_suz%2Fmw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 3 ms to seed version buckets with highest version 1507921117288857600
   [junit4]   2> 1281406 INFO  (coreLoadExecutor-933-thread-1-processing-n:127.0.0.1:55380_yc_suz%2Fmw) [n:127.0.0.1:55380_yc_suz%2Fmw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 1281407 INFO  (coreLoadExecutor-933-thread-1-processing-n:127.0.0.1:55380_yc_suz%2Fmw) [n:127.0.0.1:55380_yc_suz%2Fmw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 1281407 INFO  (coreZkRegister-927-thread-1-processing-n:127.0.0.1:55380_yc_suz%2Fmw x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:55380_yc_suz%2Fmw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:https://127.0.0.1:55380/yc_suz/mw collection:control_collection shard:shard1
   [junit4]   2> 1281408 INFO  (coreZkRegister-927-thread-1-processing-n:127.0.0.1:55380_yc_suz%2Fmw x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:55380_yc_suz%2Fmw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 1281412 INFO  (coreZkRegister-927-thread-1-processing-n:127.0.0.1:55380_yc_suz%2Fmw x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:55380_yc_suz%2Fmw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94245069702037509-core_node1-n_0000000000
   [junit4]   2> 1281413 INFO  (coreZkRegister-927-thread-1-processing-n:127.0.0.1:55380_yc_suz%2Fmw x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:55380_yc_suz%2Fmw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 1281414 INFO  (zkCallback-199-thread-1-processing-n:127.0.0.1:55380_yc_suz%2Fmw) [n:127.0.0.1:55380_yc_suz%2Fmw    ] o.a.s.c.DistributedQueue NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 1281414 INFO  (coreZkRegister-927-thread-1-processing-n:127.0.0.1:55380_yc_suz%2Fmw x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:55380_yc_suz%2Fmw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1281414 INFO  (coreZkRegister-927-thread-1-processing-n:127.0.0.1:55380_yc_suz%2Fmw x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:55380_yc_suz%2Fmw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1281415 INFO  (coreZkRegister-927-thread-1-processing-n:127.0.0.1:55380_yc_suz%2Fmw x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:55380_yc_suz%2Fmw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:55380/yc_suz/mw/collection1/
   [junit4]   2> 1281415 INFO  (coreZkRegister-927-thread-1-processing-n:127.0.0.1:55380_yc_suz%2Fmw x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:55380_yc_suz%2Fmw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 1281415 INFO  (coreZkRegister-927-thread-1-processing-n:127.0.0.1:55380_yc_suz%2Fmw x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:55380_yc_suz%2Fmw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy https://127.0.0.1:55380/yc_suz/mw/collection1/ has no replicas
   [junit4]   2> 1281415 INFO  (coreZkRegister-927-thread-1-processing-n:127.0.0.1:55380_yc_suz%2Fmw x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:55380_yc_suz%2Fmw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:55380/yc_suz/mw/collection1/ shard1
   [junit4]   2> 1281415 INFO  (OverseerStateUpdate-94245069702037509-127.0.0.1:55380_yc_suz%2Fmw-n_0000000000) [n:127.0.0.1:55380_yc_suz%2Fmw    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 1281415 INFO  (coreZkRegister-927-thread-1-processing-n:127.0.0.1:55380_yc_suz%2Fmw x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:55380_yc_suz%2Fmw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 1281419 INFO  (zkCallback-199-thread-1-processing-n:127.0.0.1:55380_yc_suz%2Fmw) [n:127.0.0.1:55380_yc_suz%2Fmw    ] o.a.s.c.DistributedQueue NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 1281420 INFO  (OverseerStateUpdate-94245069702037509-127.0.0.1:55380_yc_suz%2Fmw-n_0000000000) [n:127.0.0.1:55380_yc_suz%2Fmw    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"https://127.0.0.1:55380/yc_suz/mw",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 1
   [junit4]   2> 1281522 INFO  (zkCallback-199-thread-1-processing-n:127.0.0.1:55380_yc_suz%2Fmw) [n:127.0.0.1:55380_yc_suz%2Fmw    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1281523 INFO  (zkCallback-199-thread-1-processing-n:127.0.0.1:55380_yc_suz%2Fmw) [n:127.0.0.1:55380_yc_suz%2Fmw    ] o.a.s.c.c.ZkStateReader Updated cluster state version to 2
   [junit4]   2> 1281570 INFO  (coreZkRegister-927-thread-1-processing-n:127.0.0.1:55380_yc_suz%2Fmw x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:55380_yc_suz%2Fmw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are https://127.0.0.1:55380/yc_suz/mw/collection1/ and leader is https://127.0.0.1:55380/yc_suz/mw/collection1/
   [junit4]   2> 1281571 INFO  (coreZkRegister-927-thread-1-processing-n:127.0.0.1:55380_yc_suz%2Fmw x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:55380_yc_suz%2Fmw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:55380/yc_suz/mw
   [junit4]   2> 1281571 INFO  (coreZkRegister-927-thread-1-processing-n:127.0.0.1:55380_yc_suz%2Fmw x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:55380_yc_suz%2Fmw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1281571 INFO  (coreZkRegister-927-thread-1-processing-n:127.0.0.1:55380_yc_suz%2Fmw x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:55380_yc_suz%2Fmw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 1281571 INFO  (coreZkRegister-927-thread-1-processing-n:127.0.0.1:55380_yc_suz%2Fmw x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:55380_yc_suz%2Fmw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1281572 INFO  (zkCallback-199-thread-1-processing-n:127.0.0.1:55380_yc_suz%2Fmw) [n:127.0.0.1:55380_yc_suz%2Fmw    ] o.a.s.c.DistributedQueue NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 1281573 INFO  (OverseerStateUpdate-94245069702037509-127.0.0.1:55380_yc_suz%2Fmw-n_0000000000) [n:127.0.0.1:55380_yc_suz%2Fmw    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:55380/yc_suz/mw",
   [junit4]   2>   "node_name":"127.0.0.1:55380_yc_suz%2Fmw",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"} current state version: 2
   [junit4]   2> 1281574 INFO  (OverseerStateUpdate-94245069702037509-127.0.0.1:55380_yc_suz%2Fmw-n_0000000000) [n:127.0.0.1:55380_yc_suz%2Fmw    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:55380/yc_suz/mw",
   [junit4]   2>   "node_name":"127.0.0.1:55380_yc_suz%2Fmw",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 1281677 INFO  (zkCallback-199-thread-1-processing-n:127.0.0.1:55380_yc_suz%2Fmw) [n:127.0.0.1:55380_yc_suz%2Fmw    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1281679 INFO  (zkCallback-199-thread-1-processing-n:127.0.0.1:55380_yc_suz%2Fmw) [n:127.0.0.1:55380_yc_suz%2Fmw    ] o.a.s.c.c.ZkStateReader Updated cluster state version to 3
   [junit4]   2> 1281722 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1281723 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1281725 INFO  (zkCallback-201-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6ddfd89e name:ZooKeeperConnection Watcher:127.0.0.1:43713/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1281726 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1281727 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1281727 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1281730 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 1281730 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with stateFormat=2
   [junit4]   2> 1281730 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1281730 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1281737 INFO  (zkCallback-202-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@740bb9a5 name:ZooKeeperConnection Watcher:127.0.0.1:43713/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1281737 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1281737 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1281739 INFO  (zkCallback-199-thread-1-processing-n:127.0.0.1:55380_yc_suz%2Fmw) [n:127.0.0.1:55380_yc_suz%2Fmw    ] o.a.s.c.DistributedQueue NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 1281740 INFO  (OverseerStateUpdate-94245069702037509-127.0.0.1:55380_yc_suz%2Fmw-n_0000000000) [n:127.0.0.1:55380_yc_suz%2Fmw    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"create",
   [junit4]   2>   "name":"collection1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "stateFormat":"2"} current state version: 3
   [junit4]   2> 1281740 INFO  (OverseerStateUpdate-94245069702037509-127.0.0.1:55380_yc_suz%2Fmw-n_0000000000) [n:127.0.0.1:55380_yc_suz%2Fmw    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 1281853 INFO  (OverseerStateUpdate-94245069702037509-127.0.0.1:55380_yc_suz%2Fmw-n_0000000000) [n:127.0.0.1:55380_yc_suz%2Fmw    ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/collection1/state.json
   [junit4]   2> 1281855 INFO  (zkCallback-201-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1281855 INFO  (zkCallback-199-thread-1-processing-n:127.0.0.1:55380_yc_suz%2Fmw) [n:127.0.0.1:55380_yc_suz%2Fmw    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1281857 INFO  (zkCallback-201-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated cluster state version to 4
   [junit4]   2> 1281857 INFO  (zkCallback-199-thread-1-processing-n:127.0.0.1:55380_yc_suz%2Fmw) [n:127.0.0.1:55380_yc_suz%2Fmw    ] o.a.s.c.c.ZkStateReader Updated cluster state version to 4
   [junit4]   2> 1282033 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_F6FA78EB9F8891F-001/shard-1-001/cores/collection1
   [junit4]   2> 1282034 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_F6FA78EB9F8891F-001/shard-1-001
   [junit4]   2> 1282035 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [    ] o.e.j.s.Server jetty-9.2.11.v20150529
   [junit4]   2> 1282039 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@7b92ba32{/yc_suz/mw,null,AVAILABLE}
   [junit4]   2> 1282047 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [    ] o.e.j.s.ServerConnector Started ServerConnector@13cbd074{SSL-http/1.1}{127.0.0.1:41351}
   [junit4]   2> 1282047 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [    ] o.e.j.s.Server Started @1285622ms
   [junit4]   2> 1282047 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_F6FA78EB9F8891F-001/tempDir-001/jetty1, solrconfig=solrconfig.xml, hostContext=/yc_suz/mw, hostPort=41351, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_F6FA78EB9F8891F-001/shard-1-001/cores}
   [junit4]   2> 1282049 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@4e0e2f2a
   [junit4]   2> 1282050 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_F6FA78EB9F8891F-001/shard-1-001/'
   [junit4]   2> 1282065 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1282066 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1282068 INFO  (zkCallback-203-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2b19284a name:ZooKeeperConnection Watcher:127.0.0.1:43713/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1282068 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1282069 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1282070 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1282071 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_F6FA78EB9F8891F-001/shard-1-001/solr.xml
   [junit4]   2> 1282076 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_F6FA78EB9F8891F-001/shard-1-001/cores
   [junit4]   2> 1282076 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [    ] o.a.s.c.CoreContainer New CoreContainer 1640807145
   [junit4]   2> 1282076 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_F6FA78EB9F8891F-001/shard-1-001/]
   [junit4]   2> 1282077 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_F6FA78EB9F8891F-001/shard-1-001/lib
   [junit4]   2> 1282077 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_F6FA78EB9F8891F-001/shard-1-001/lib).
   [junit4]   2> 1282084 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1282087 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1282088 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1282088 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1282088 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:43713/solr
   [junit4]   2> 1282089 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1282089 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1282089 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1282091 INFO  (zkCallback-205-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@11b84204 name:ZooKeeperConnection Watcher:127.0.0.1:43713 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1282092 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1282092 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1282095 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [n:127.0.0.1:41351_yc_suz%2Fmw    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1282101 INFO  (zkCallback-206-thread-1-processing-n:127.0.0.1:41351_yc_suz%2Fmw) [n:127.0.0.1:41351_yc_suz%2Fmw    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5c8c5d3 name:ZooKeeperConnection Watcher:127.0.0.1:43713/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1282101 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [n:127.0.0.1:41351_yc_suz%2Fmw    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1282107 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [n:127.0.0.1:41351_yc_suz%2Fmw    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1283111 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [n:127.0.0.1:41351_yc_suz%2Fmw    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:41351_yc_suz%2Fmw
   [junit4]   2> 1283112 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [n:127.0.0.1:41351_yc_suz%2Fmw    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:41351_yc_suz%2Fmw
   [junit4]   2> 1283114 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [n:127.0.0.1:41351_yc_suz%2Fmw    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1283115 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [n:127.0.0.1:41351_yc_suz%2Fmw    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94245069702037514-127.0.0.1:41351_yc_suz%2Fmw-n_0000000001
   [junit4]   2> 1283116 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [n:127.0.0.1:41351_yc_suz%2Fmw    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/94245069702037509-127.0.0.1:55380_yc_suz%2Fmw-n_0000000000 to know if I could be the leader
   [junit4]   2> 1283116 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [n:127.0.0.1:41351_yc_suz%2Fmw    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1283116 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [n:127.0.0.1:41351_yc_suz%2Fmw    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1283117 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [n:127.0.0.1:41351_yc_suz%2Fmw    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_F6FA78EB9F8891F-001/shard-1-001/cores
   [junit4]   2> 1283118 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [n:127.0.0.1:41351_yc_suz%2Fmw    ] o.a.s.c.SolrCore Created CoreDescriptor: {name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, loadOnStartup=true, configSetProperties=configsetprops.json, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_F6FA78EB9F8891F-001/shard-1-001/cores/collection1, collection=collection1, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_F6FA78EB9F8891F-001/shard-1-001/cores/collection1/, coreNodeName=, dataDir=data/, shard=}
   [junit4]   2> 1283118 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [n:127.0.0.1:41351_yc_suz%2Fmw    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_F6FA78EB9F8891F-001/shard-1-001/cores/collection1/
   [junit4]   2> 1283118 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [n:127.0.0.1:41351_yc_suz%2Fmw    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 1283120 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [n:127.0.0.1:41351_yc_suz%2Fmw    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1
   [junit4]   2> 1283120 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F6FA78EB9F8891F]) [n:127.0.0.1:41351_yc_suz%2Fmw    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1283121 INFO  (coreLoadExecutor-944-thread-1-processing-n:127.0.0.1:41351_yc_suz%2Fmw) [n:127.0.0.1:41351_yc_suz%2Fmw c:collection1   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 1283121 INFO  (coreLoadExecutor-944-thread-1-processing-n:127.0.0.1:41351_yc_suz%2Fmw) [n:127.0.0.1:41351_yc_suz%2Fmw c:collection1   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1283122 INFO  (zkCallback-199-thread-1-processing-n:127.0.0.1:55380_yc_suz%2Fmw) [n:127.0.0.1:55380_yc_suz%2Fmw    ] o.a.s.c.DistributedQueue NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 1283123 INFO  (coreLoadExecutor-944-thread-1-processing-n:127.0.0.1:41351_yc_suz%2Fmw) [n:127.0.0.1:41351_yc_suz%2Fmw c:collection1   x:collection1] o.a.s.c.ZkController Registering watch for external collection collection1
   [junit4]   2> 1283123 INFO  (coreLoadExecutor-944-thread-1-processing-n:127.0.0.1:41351_yc_suz%2Fmw) [n:127.0.0.1:41351_yc_suz%2Fmw c:collection1   x:collection1] o.a.s.c.c.ZkStateReader addZkWatch collection1
   [junit4]   2> 1283123 INFO  (OverseerStateUpdate-94245069702037509-127.0.0.1:55380_yc_suz%2Fmw-n_0000000000) [n:127.0.0.1:55380_yc_suz%2Fmw    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:41351/yc_suz/mw",
   [junit4]   2>   "node_name":"127.0.0.1:41351_yc_suz%2Fmw",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "operation":"state"} current state version: 4
   [junit4]   2> 1283123 INFO  (coreLoadExecutor-944-thread-1-processing-n:127.0.0.1:41351_yc_suz%2Fmw) [n:127.0.0.1:41351_yc_suz%2Fmw c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Updating collection state at /collections/collection1/state.json from ZooKeeper... 
   [junit4]   2> 1283124 INFO  (OverseerStateUpdate-94245069702037509-127.0.0.1:55380_yc_suz%2Fmw-n_0000000000) [n:127.0.0.1:55380_yc_suz%2Fmw    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:41351/yc_suz/mw",
   [junit4]   2>   "node_name":"127.0.0.1:41351_yc_suz%2Fmw",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 1283124 INFO  (OverseerStateUpdate-94245069702037509-127.0.0.1:55380_yc_suz%2Fmw-n_0000000000) [n:127.0.0.1:55380_yc_suz%2Fmw    ] o.a.s.c.o.ReplicaMutator Collection already exists with numShards=2
   [junit4]   2> 1283124 INFO  (OverseerStateUpdate-94245069702037509-127.0.0.1:55380_yc_suz%2Fmw-n_0000000000) [n:127.0.0.1:55380_yc_suz%2Fmw    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2
   [junit4]   2> 1283124 INFO  (coreLoadExecutor-944-thread-1-processing-n:127.0.0.1:41351_yc_suz%2Fmw) [n:127.0.0.1:41351_yc_suz%2Fmw c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Updating data for collection1 to ver 0 
   [junit4]   2> 1283125 INFO  (coreLoadExecutor-944-thread-1-processing-n:127.0.0.1:41351_yc_suz%2Fmw) [n:127.0.0.1:41351_yc_suz%2Fmw c:collection1   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 1283227 INFO  (OverseerStateUpdate-94245069702037509-127.0.0.1:55380_yc_suz%2Fmw-n_0000000000) [n:127.0.0.1:55380_yc_suz%2Fmw    ] o.a.s.c.o.ZkStateWriter going to update_collection /collections/collection1/state.json version: 0
   [junit4]   2> 1283228 INFO  (zkCallback-206-thread-1-processing-n:127.0.0.1:41351_yc_suz%2Fmw) [n:127.0.0.1:41351_yc_suz%2Fmw    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json for collection collection1 has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1283229 INFO  (zkCallback-206-thread-1-processing-n:127.0.0.1:41351_yc_suz%2Fmw) [n:127.0.0.1:41351_yc_suz%2Fmw    ] o.a.s.c.c.ZkStateReader Updating data for collection1 to ver 1 
   [junit4]   2> 1284125 INFO  (coreLoadExecutor-944-thread-1-processing-n:127.0.0.1:41351_yc_suz%2Fmw) [n:127.0.0.1:41351_yc_suz%2Fmw c:collection1   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1284125 INFO  (coreLoadExecutor-944-thread-1-processing-n:127.0.0.1:41351_yc_suz%2Fmw) [n:127.0.0.1:41351_yc_suz%2Fmw c:collection1   x:collection1] o.a.s.c.ZkController Check for collection zkNode:collection1
   [junit4]   2> 1284126 INFO  (coreLoadExecutor-944-thread-1-processing-n:127.0.0.1:41351_yc_suz%2Fmw) [n:127.0.0.1:41351_yc_suz%2Fmw c:collection1   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 1284126 INFO  (coreLoadExecutor-944-thread-1-processing-n:127.0.0.1:41351_yc_suz%2Fmw) [n:127.0.0.1:41351_yc_suz%2Fmw c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/collection1
   [junit4]   2> 1284127 INFO  (coreLoadExecutor-944-thread-1-processing-n:127.0.0.1:41351_yc_suz%2Fmw) [n:127.0.0.1:41351_yc_suz%2Fmw c:collection1   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1284127 INFO  (coreLoadExecutor-944-thread-1-processing-n:127.0.0.1:41351_yc_suz%2Fmw) [n:127.0.0.1:41351_yc_suz%2Fmw c:collection1   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_F6FA78EB9F8891F-001/shard-1-001/cores/collection1/'
   [junit4]   2> 1284140 INFO  (coreLoadExecutor-944-thread-1-processing-n:127.0.0.1:41351_yc_suz%2Fmw) [n:127.0.0.1:41351_yc_suz%2Fmw c:collection1   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 1284145 INFO  (coreLoadExecutor-944-thread-1-processing-n:127.0.0.1:41351_yc_suz%2Fmw) [n:127.0.0.1:41351_yc_suz%2Fmw c:collection1   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 1284150 INFO  (coreLoadExecutor-944-thread-1-processing-n:127.0.0.1:41351_yc_suz%2Fmw) [n:127.0.0.1:41351_yc_suz%2Fmw c:collection1   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.0.0
   [junit4]   2> 1284160 INFO  (coreLoadExecutor-944-thread-1-processing-n:127.0.0.1:41351_yc_suz%2Fmw) [n:127.0.0.1:41351_yc_suz%2Fmw c:collection1   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1284161 INFO  (coreLoadExecutor-944-thread-1-processing-n:127.0.0.1:41351_yc_suz%2Fmw) [n:127.0.0.1:41351_yc_suz%2Fmw c:collection1   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 1284166 INFO  (coreLoadExecutor-944-thread-1-processing-n:127.0.0.1:41351_yc_suz%2Fmw) [n:127.0.0.1:41351_yc_suz%2Fmw c:collection1   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1284262 INFO  (coreLoadExecutor-944-thread-1-processing-n:127.0.0.1:41351_yc_suz%2Fmw) [n:127.0.0.1:41351_yc_suz%2Fmw c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 1284269 INFO  (coreLoadExecutor-944-thread-1-processing-n:127.0.0.1:41351_yc_suz%2Fmw) [n:127.0.0.1:41351_yc_suz%2Fmw c:collection1   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 1284270 INFO  (coreLoadExecutor-944-thread-1-processing-n:127.0.0.1:41351_yc_suz%2Fmw) [n:127.0.0.1:41351_yc_suz%2Fmw c:collection1   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 1284281 INFO  (coreLoadExecutor-944-thread-1-processing-n:127.0.0.1:41351_yc_suz%2Fmw) [n:127.0.0.1:41351_yc_suz%2Fmw c:collection1   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1284284 INFO  (coreLoadExecutor-944-thread-1-processing-n:127.0.0.1:41351_yc_suz%2Fmw) [n:127.0.0.1:41351_yc_suz%2Fmw c:collection1   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1284285 INFO  (coreLoadExecutor-944-thread-1-processing-n:127.0.0.1:41351_yc_suz%2Fmw) [n:127.0.0.1:41351_yc_suz%2Fmw c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1284286 WARN  (coreLoadExecutor-944-thread-1-processing-n:127.0.0.1:41351_yc_suz%2Fmw) [n:127.0.0.1:41351_yc_suz%2Fmw c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 1284286 WARN  (coreLoadExecutor-944-thread-1-processing-n:127.0.0.1:41351_yc_suz%2Fmw) [n:127.0.0.1:41351_yc_suz%2Fmw c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 1284286 INFO  (coreLoadExecutor-944-thread-1-processing-n:127.0.0.1:41351_yc_suz%2Fmw) [n:127.0.0.1:41351_yc_suz%2Fmw c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1284287 WARN  (coreLoadExecutor-944-thread-1-processing-n:127.0.0.1:41351_yc_suz%2Fmw) [n:127.0.0.1:41351_yc_suz%2Fmw c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 1284287 WARN  (coreLoa

[...truncated too long message...]

kins-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build.xml:227: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/common-build.xml:517: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/common-build.xml:1429: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/common-build.xml:984: At least one slave process threw an exception, first: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.

Total time: 393 minutes 20 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
No prior successful build to compare, so performing full copy of artifacts
ERROR: Failed to archive {README.txt=README.txt, solr/build/solr-core/test/temp/junit4-J2-20150728_021915_691.events=solr/build/solr-core/test/temp/junit4-J2-20150728_021915_691.events, solr/build/solr-core/test/temp/junit4-J1-20150728_021915_690.events=solr/build/solr-core/test/temp/junit4-J1-20150728_021915_690.events, solr/build/solr-core/test/temp/junit4-J0-20150728_021915_690.events=solr/build/solr-core/test/temp/junit4-J0-20150728_021915_690.events} due to internal error; falling back to full archiving
java.lang.IllegalArgumentException: Negative time
	at java.io.File.setLastModified(File.java:1421)
	at hudson.FilePath.readFromTar(FilePath.java:2093)
	at hudson.FilePath.copyRecursiveTo(FilePath.java:2013)
	at jenkins.model.StandardArtifactManager.archive(StandardArtifactManager.java:61)
	at com.cloudbees.jenkins.plugins.jsync.archiver.JSyncArtifactManager.remoteSync(JSyncArtifactManager.java:107)
	at com.cloudbees.jenkins.plugins.jsync.archiver.JSyncArtifactManager.archive(JSyncArtifactManager.java:68)
	at hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:140)
	at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
	at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:756)
	at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:720)
	at hudson.model.Build$BuildExecution.post2(Build.java:182)
	at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:669)
	at hudson.model.Run.execute(Run.java:1731)
	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
	at hudson.model.ResourceController.execute(ResourceController.java:88)
	at hudson.model.Executor.run(Executor.java:232)
ERROR: Failed to archive artifacts: **/*.events,heapdumps/**,**/hs_err_pid*,README.txt
java.io.IOException: java.io.IOException: Failed to extract /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/transfer of 4 files
	at hudson.FilePath.readFromTar(FilePath.java:2101)
	at hudson.FilePath.copyRecursiveTo(FilePath.java:2013)
	at jenkins.model.StandardArtifactManager.archive(StandardArtifactManager.java:61)
	at com.cloudbees.jenkins.plugins.jsync.archiver.JSyncArtifactManager.archive(JSyncArtifactManager.java:72)
	at hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:140)
	at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
	at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:756)
	at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:720)
	at hudson.model.Build$BuildExecution.post2(Build.java:182)
	at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:669)
	at hudson.model.Run.execute(Run.java:1731)
	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
	at hudson.model.ResourceController.execute(ResourceController.java:88)
	at hudson.model.Executor.run(Executor.java:232)
Caused by: java.io.FileNotFoundException: /x1/jenkins/jenkins-home/jobs/Lucene-Solr-NightlyTests-trunk/builds/2015-07-28_04-34-48/archive/@362aa30d) [    ] o.a.h.h.s.d.DataNode 127.0.0.1:48869:DataXceiverServer: %0Ajava.nio.channels.Close (Permission denied)
	at java.io.FileOutputStream.open(Native Method)
	at java.io.FileOutputStream.<init>(FileOutputStream.java:221)
	at java.io.FileOutputStream.<init>(FileOutputStream.java:171)
	at hudson.util.IOUtils.copy(IOUtils.java:38)
	at hudson.FilePath.readFromTar(FilePath.java:2091)
	... 13 more

	at hudson.FilePath.copyRecursiveTo(FilePath.java:2020)
	at jenkins.model.StandardArtifactManager.archive(StandardArtifactManager.java:61)
	at com.cloudbees.jenkins.plugins.jsync.archiver.JSyncArtifactManager.archive(JSyncArtifactManager.java:72)
	at hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:140)
	at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
	at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:756)
	at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:720)
	at hudson.model.Build$BuildExecution.post2(Build.java:182)
	at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:669)
	at hudson.model.Run.execute(Run.java:1731)
	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
	at hudson.model.ResourceController.execute(ResourceController.java:88)
	at hudson.model.Executor.run(Executor.java:232)
Caused by: java.util.concurrent.ExecutionException: java.io.IOException: Pipe is already closed
	at hudson.remoting.Channel$3.adapt(Channel.java:775)
	at hudson.remoting.Channel$3.adapt(Channel.java:770)
	at hudson.remoting.FutureAdapter.get(FutureAdapter.java:59)
	at hudson.FilePath.copyRecursiveTo(FilePath.java:2016)
	... 12 more
Caused by: java.io.IOException: Pipe is already closed
	at hudson.remoting.PipeWindow.checkDeath(PipeWindow.java:108)
	at hudson.remoting.PipeWindow$Real.get(PipeWindow.java:203)
	at hudson.remoting.ProxyOutputStream._write(ProxyOutputStream.java:141)
	at hudson.remoting.ProxyOutputStream.write(ProxyOutputStream.java:109)
	at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
	at java.io.BufferedOutputStream.write(BufferedOutputStream.java:126)
	at com.jcraft.jzlib.DeflaterOutputStream.deflate(DeflaterOutputStream.java:144)
	at com.jcraft.jzlib.DeflaterOutputStream.write(DeflaterOutputStream.java:102)
	at java.io.BufferedOutputStream.write(BufferedOutputStream.java:122)
	at org.apache.tools.tar.TarBuffer.writeBlock(TarBuffer.java:410)
	at org.apache.tools.tar.TarBuffer.writeRecord(TarBuffer.java:351)
	at hudson.org.apache.tools.tar.TarOutputStream.writeEOFRecord(TarOutputStream.java:356)
	at hudson.org.apache.tools.tar.TarOutputStream.finish(TarOutputStream.java:137)
	at hudson.org.apache.tools.tar.TarOutputStream.close(TarOutputStream.java:149)
	at hudson.util.io.TarArchiver.close(TarArchiver.java:125)
	at hudson.FilePath.writeToTar(FilePath.java:2067)
	at hudson.FilePath.access$1000(FilePath.java:178)
	at hudson.FilePath$41.invoke(FilePath.java:2006)
	at hudson.FilePath$41.invoke(FilePath.java:2002)
	at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2474)
	at hudson.remoting.UserRequest.perform(UserRequest.java:118)
	at hudson.remoting.UserRequest.perform(UserRequest.java:48)
	at hudson.remoting.Request$2.run(Request.java:328)
	at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.IOException: Pipe is already closed
	at hudson.remoting.FastPipedOutputStream.write(FastPipedOutputStream.java:153)
	at hudson.remoting.FastPipedOutputStream.write(FastPipedOutputStream.java:137)
	at hudson.remoting.ProxyOutputStream$Chunk$1.run(ProxyOutputStream.java:264)
	at hudson.remoting.PipeWriter$1.run(PipeWriter.java:158)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:111)
	at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
	at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46)
	... 4 more
Caused by: hudson.remoting.FastPipedInputStream$ClosedBy: The pipe was closed at...
	at hudson.remoting.FastPipedInputStream.close(FastPipedInputStream.java:112)
	at java.io.FilterInputStream.close(FilterInputStream.java:181)
	at com.jcraft.jzlib.InflaterInputStream.close(InflaterInputStream.java:168)
	at org.apache.tools.tar.TarBuffer.close(TarBuffer.java:456)
	at hudson.org.apache.tools.tar.TarInputStream.close(TarInputStream.java:110)
	at hudson.FilePath.readFromTar(FilePath.java:2108)
	at hudson.FilePath.copyRecursiveTo(FilePath.java:2013)
	at jenkins.model.StandardArtifactManager.archive(StandardArtifactManager.java:61)
	at com.cloudbees.jenkins.plugins.jsync.archiver.JSyncArtifactManager.archive(JSyncArtifactManager.java:72)
	at hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:140)
	at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
	at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:756)
	at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:720)
	at hudson.model.Build$BuildExecution.post2(Build.java:182)
	at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:669)
	at hudson.model.Run.execute(Run.java:1731)
	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
	at hudson.model.ResourceController.execute(ResourceController.java:88)
	at hudson.model.Executor.run(Executor.java:232)
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-NightlyTests-trunk - Build # 748 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-trunk/748/

1 tests failed.
REGRESSION:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Error from server at http://127.0.0.1:43373: Error CREATEing SolrCore 'halfcollection_shard1_replica1': Error reading cluster properties Caused by: KeeperErrorCode = Session expired for /clusterprops.json

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:43373: Error CREATEing SolrCore 'halfcollection_shard1_replica1': Error reading cluster properties Caused by: KeeperErrorCode = Session expired for /clusterprops.json
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:560)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:234)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:226)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.makeRequest(CollectionsAPIDistributedZkTest.java:301)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testErrorHandling(CollectionsAPIDistributedZkTest.java:418)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test(CollectionsAPIDistributedZkTest.java:168)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:497)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:963)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 11259 lines...]
   [junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_7B9D114E4CC14E06-001/init-core-data-001
   [junit4]   2> 1549435 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[7B9D114E4CC14E06]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false)
   [junit4]   2> 1549436 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[7B9D114E4CC14E06]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   2> 1549438 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 1549438 INFO  (Thread-2672) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1549439 INFO  (Thread-2672) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 1549538 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [    ] o.a.s.c.ZkTestServer start zk server on port:36981
   [junit4]   2> 1549539 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1549540 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1549542 INFO  (zkCallback-550-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@165006ab name:ZooKeeperConnection Watcher:127.0.0.1:36981 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1549543 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1549543 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1549543 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 1549545 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1549546 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1549548 INFO  (zkCallback-551-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@9f98e87 name:ZooKeeperConnection Watcher:127.0.0.1:36981/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1549548 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1549549 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1549549 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 1549550 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 1549552 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 1549553 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 1549555 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 1549555 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 1549558 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 1549558 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 1549560 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1549560 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1549562 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 1549562 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 1549564 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 1549564 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 1549566 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 1549566 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 1549568 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 1549568 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 1549570 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1549570 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1549572 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1549573 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1549574 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 1549575 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 1549576 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 1549577 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 1549925 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_7B9D114E4CC14E06-001/control-001/cores/collection1
   [junit4]   2> 1549927 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [    ] o.e.j.s.Server jetty-9.2.11.v20150529
   [junit4]   2> 1549929 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@60a5632b{/,null,AVAILABLE}
   [junit4]   2> 1549938 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [    ] o.e.j.s.ServerConnector Started ServerConnector@4b809595{HTTP/1.1}{127.0.0.1:?????}
   [junit4]   2> 1549938 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [    ] o.e.j.s.Server Started @???????ms
   [junit4]   2> 1549938 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_7B9D114E4CC14E06-001/tempDir-001/control/data, hostContext=/, hostPort=40631, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_7B9D114E4CC14E06-001/control-001/cores}
   [junit4]   2> 1549939 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@4e0e2f2a
   [junit4]   2> 1549939 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_7B9D114E4CC14E06-001/control-001/'
   [junit4]   2> 1549955 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1549956 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1549958 INFO  (zkCallback-552-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@70ee7ee3 name:ZooKeeperConnection Watcher:127.0.0.1:36981/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1549958 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1549958 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1549960 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1549960 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_7B9D114E4CC14E06-001/control-001/solr.xml
   [junit4]   2> 1549966 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_7B9D114E4CC14E06-001/control-001/cores
   [junit4]   2> 1549966 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [    ] o.a.s.c.CoreContainer New CoreContainer 1560753052
   [junit4]   2> 1549966 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_7B9D114E4CC14E06-001/control-001/]
   [junit4]   2> 1549966 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_7B9D114E4CC14E06-001/control-001/lib
   [junit4]   2> 1549966 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_7B9D114E4CC14E06-001/control-001/lib).
   [junit4]   2> 1549974 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1549975 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1549976 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1549976 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1549976 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:36981/solr
   [junit4]   2> 1549976 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1549977 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1549977 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1549981 INFO  (zkCallback-554-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5f46a537 name:ZooKeeperConnection Watcher:127.0.0.1:36981 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1549982 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1549983 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1549985 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [n:127.0.0.1:40631_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1549986 INFO  (zkCallback-555-thread-1-processing-n:127.0.0.1:40631_) [n:127.0.0.1:40631_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3f3d4787 name:ZooKeeperConnection Watcher:127.0.0.1:36981/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1549986 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [n:127.0.0.1:40631_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1549992 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [n:127.0.0.1:40631_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 1549994 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [n:127.0.0.1:40631_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 1549996 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [n:127.0.0.1:40631_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 1549998 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [n:127.0.0.1:40631_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 1550000 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [n:127.0.0.1:40631_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 1550002 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [n:127.0.0.1:40631_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 1550003 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [n:127.0.0.1:40631_    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 1550005 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [n:127.0.0.1:40631_    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 1550006 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [n:127.0.0.1:40631_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:40631_
   [junit4]   2> 1550006 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [n:127.0.0.1:40631_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:40631_
   [junit4]   2> 1550008 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [n:127.0.0.1:40631_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 1550013 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [n:127.0.0.1:40631_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 1550020 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [n:127.0.0.1:40631_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1550021 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [n:127.0.0.1:40631_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94239362304180228-127.0.0.1:40631_-n_0000000000
   [junit4]   2> 1550022 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [n:127.0.0.1:40631_    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:40631_
   [junit4]   2> 1550022 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [n:127.0.0.1:40631_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 1550023 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [n:127.0.0.1:40631_    ] o.a.s.c.Overseer Overseer (id=94239362304180228-127.0.0.1:40631_-n_0000000000) starting
   [junit4]   2> 1550025 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [n:127.0.0.1:40631_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 1550030 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [n:127.0.0.1:40631_    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 1550031 INFO  (OverseerCollectionProcessor-94239362304180228-127.0.0.1:40631_-n_0000000000) [n:127.0.0.1:40631_    ] o.a.s.c.OverseerCollectionProcessor Process current queue of collection creations
   [junit4]   2> 1550031 INFO  (OverseerStateUpdate-94239362304180228-127.0.0.1:40631_-n_0000000000) [n:127.0.0.1:40631_    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 1550037 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [n:127.0.0.1:40631_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1550039 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [n:127.0.0.1:40631_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1550040 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [n:127.0.0.1:40631_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1550041 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [n:127.0.0.1:40631_    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_7B9D114E4CC14E06-001/control-001/cores
   [junit4]   2> 1550042 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [n:127.0.0.1:40631_    ] o.a.s.c.SolrCore Created CoreDescriptor: {name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, loadOnStartup=true, configSetProperties=configsetprops.json, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_7B9D114E4CC14E06-001/control-001/cores/collection1, collection=control_collection, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_7B9D114E4CC14E06-001/control-001/cores/collection1/, coreNodeName=, dataDir=data/, shard=}
   [junit4]   2> 1550042 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [n:127.0.0.1:40631_    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_7B9D114E4CC14E06-001/control-001/cores/collection1/
   [junit4]   2> 1550042 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [n:127.0.0.1:40631_    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 1550043 INFO  (coreLoadExecutor-2140-thread-1-processing-n:127.0.0.1:40631_) [n:127.0.0.1:40631_ c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 1550043 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [n:127.0.0.1:40631_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1
   [junit4]   2> 1550043 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [n:127.0.0.1:40631_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1550043 INFO  (coreLoadExecutor-2140-thread-1-processing-n:127.0.0.1:40631_) [n:127.0.0.1:40631_ c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1550044 INFO  (zkCallback-555-thread-1-processing-n:127.0.0.1:40631_) [n:127.0.0.1:40631_    ] o.a.s.c.DistributedQueue NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 1550044 INFO  (coreLoadExecutor-2140-thread-1-processing-n:127.0.0.1:40631_) [n:127.0.0.1:40631_ c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 1550045 INFO  (OverseerStateUpdate-94239362304180228-127.0.0.1:40631_-n_0000000000) [n:127.0.0.1:40631_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:40631",
   [junit4]   2>   "node_name":"127.0.0.1:40631_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 1550046 INFO  (OverseerStateUpdate-94239362304180228-127.0.0.1:40631_-n_0000000000) [n:127.0.0.1:40631_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:40631",
   [junit4]   2>   "node_name":"127.0.0.1:40631_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 1550046 INFO  (OverseerStateUpdate-94239362304180228-127.0.0.1:40631_-n_0000000000) [n:127.0.0.1:40631_    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 1550046 INFO  (OverseerStateUpdate-94239362304180228-127.0.0.1:40631_-n_0000000000) [n:127.0.0.1:40631_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 1550047 INFO  (zkCallback-555-thread-1-processing-n:127.0.0.1:40631_) [n:127.0.0.1:40631_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1550049 INFO  (zkCallback-555-thread-1-processing-n:127.0.0.1:40631_) [n:127.0.0.1:40631_    ] o.a.s.c.c.ZkStateReader Updated cluster state version to 1
   [junit4]   2> 1551045 INFO  (coreLoadExecutor-2140-thread-1-processing-n:127.0.0.1:40631_) [n:127.0.0.1:40631_ c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1551045 INFO  (coreLoadExecutor-2140-thread-1-processing-n:127.0.0.1:40631_) [n:127.0.0.1:40631_ c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 1551046 INFO  (coreLoadExecutor-2140-thread-1-processing-n:127.0.0.1:40631_) [n:127.0.0.1:40631_ c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 1551046 INFO  (coreLoadExecutor-2140-thread-1-processing-n:127.0.0.1:40631_) [n:127.0.0.1:40631_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 1551047 INFO  (coreLoadExecutor-2140-thread-1-processing-n:127.0.0.1:40631_) [n:127.0.0.1:40631_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1551047 INFO  (coreLoadExecutor-2140-thread-1-processing-n:127.0.0.1:40631_) [n:127.0.0.1:40631_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_7B9D114E4CC14E06-001/control-001/cores/collection1/'
   [junit4]   2> 1551067 INFO  (coreLoadExecutor-2140-thread-1-processing-n:127.0.0.1:40631_) [n:127.0.0.1:40631_ c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 1551085 INFO  (coreLoadExecutor-2140-thread-1-processing-n:127.0.0.1:40631_) [n:127.0.0.1:40631_ c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 1551089 INFO  (coreLoadExecutor-2140-thread-1-processing-n:127.0.0.1:40631_) [n:127.0.0.1:40631_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.0.0
   [junit4]   2> 1551098 INFO  (coreLoadExecutor-2140-thread-1-processing-n:127.0.0.1:40631_) [n:127.0.0.1:40631_ c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1551100 INFO  (coreLoadExecutor-2140-thread-1-processing-n:127.0.0.1:40631_) [n:127.0.0.1:40631_ c:control_collection   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 1551105 INFO  (coreLoadExecutor-2140-thread-1-processing-n:127.0.0.1:40631_) [n:127.0.0.1:40631_ c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1551200 INFO  (coreLoadExecutor-2140-thread-1-processing-n:127.0.0.1:40631_) [n:127.0.0.1:40631_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 1551207 INFO  (coreLoadExecutor-2140-thread-1-processing-n:127.0.0.1:40631_) [n:127.0.0.1:40631_ c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 1551208 INFO  (coreLoadExecutor-2140-thread-1-processing-n:127.0.0.1:40631_) [n:127.0.0.1:40631_ c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 1551222 INFO  (coreLoadExecutor-2140-thread-1-processing-n:127.0.0.1:40631_) [n:127.0.0.1:40631_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1551238 INFO  (coreLoadExecutor-2140-thread-1-processing-n:127.0.0.1:40631_) [n:127.0.0.1:40631_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1551241 INFO  (coreLoadExecutor-2140-thread-1-processing-n:127.0.0.1:40631_) [n:127.0.0.1:40631_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1551242 WARN  (coreLoadExecutor-2140-thread-1-processing-n:127.0.0.1:40631_) [n:127.0.0.1:40631_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 1551242 WARN  (coreLoadExecutor-2140-thread-1-processing-n:127.0.0.1:40631_) [n:127.0.0.1:40631_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 1551242 INFO  (coreLoadExecutor-2140-thread-1-processing-n:127.0.0.1:40631_) [n:127.0.0.1:40631_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1551243 WARN  (coreLoadExecutor-2140-thread-1-processing-n:127.0.0.1:40631_) [n:127.0.0.1:40631_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 1551243 WARN  (coreLoadExecutor-2140-thread-1-processing-n:127.0.0.1:40631_) [n:127.0.0.1:40631_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 1551244 INFO  (coreLoadExecutor-2140-thread-1-processing-n:127.0.0.1:40631_) [n:127.0.0.1:40631_ c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1
   [junit4]   2> 1551245 INFO  (coreLoadExecutor-2140-thread-1-processing-n:127.0.0.1:40631_) [n:127.0.0.1:40631_ c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 1551245 INFO  (coreLoadExecutor-2140-thread-1-processing-n:127.0.0.1:40631_) [n:127.0.0.1:40631_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 1551245 INFO  (coreLoadExecutor-2140-thread-1-processing-n:127.0.0.1:40631_) [n:127.0.0.1:40631_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_7B9D114E4CC14E06-001/control-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 1551245 INFO  (coreLoadExecutor-2140-thread-1-processing-n:127.0.0.1:40631_) [n:127.0.0.1:40631_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@699fe850
   [junit4]   2> 1551246 INFO  (coreLoadExecutor-2140-thread-1-processing-n:127.0.0.1:40631_) [n:127.0.0.1:40631_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_7B9D114E4CC14E06-001/control-001/cores/collection1/data
   [junit4]   2> 1551246 INFO  (coreLoadExecutor-2140-thread-1-processing-n:127.0.0.1:40631_) [n:127.0.0.1:40631_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_7B9D114E4CC14E06-001/control-001/cores/collection1/data/index/
   [junit4]   2> 1551246 WARN  (coreLoadExecutor-2140-thread-1-processing-n:127.0.0.1:40631_) [n:127.0.0.1:40631_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_7B9D114E4CC14E06-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 1551247 INFO  (coreLoadExecutor-2140-thread-1-processing-n:127.0.0.1:40631_) [n:127.0.0.1:40631_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_7B9D114E4CC14E06-001/control-001/cores/collection1/data/index
   [junit4]   2> 1551247 INFO  (coreLoadExecutor-2140-thread-1-processing-n:127.0.0.1:40631_) [n:127.0.0.1:40631_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=32, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.5004314197358152]
   [junit4]   2> 1551248 INFO  (coreLoadExecutor-2140-thread-1-processing-n:127.0.0.1:40631_) [n:127.0.0.1:40631_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@1b15f6e4 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@c5c1d1e),segFN=segments_1,generation=1}
   [junit4]   2> 1551248 INFO  (coreLoadExecutor-2140-thread-1-processing-n:127.0.0.1:40631_) [n:127.0.0.1:40631_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 1551248 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:40631_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_7B9D114E4CC14E06-001/control-001/cores/collection1/data/
   [junit4]   2> 1551249 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:40631_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_7B9D114E4CC14E06-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 1551253 INFO  (coreLoadExecutor-2140-thread-1-processing-n:127.0.0.1:40631_) [n:127.0.0.1:40631_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 1551253 INFO  (coreLoadExecutor-2140-thread-1-processing-n:127.0.0.1:40631_) [n:127.0.0.1:40631_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 1551254 INFO  (coreLoadExecutor-2140-thread-1-processing-n:127.0.0.1:40631_) [n:127.0.0.1:40631_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 1551254 INFO  (coreLoadExecutor-2140-thread-1-processing-n:127.0.0.1:40631_) [n:127.0.0.1:40631_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1551254 INFO  (coreLoadExecutor-2140-thread-1-processing-n:127.0.0.1:40631_) [n:127.0.0.1:40631_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1551254 INFO  (coreLoadExecutor-2140-thread-1-processing-n:127.0.0.1:40631_) [n:127.0.0.1:40631_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 1551254 INFO  (coreLoadExecutor-2140-thread-1-processing-n:127.0.0.1:40631_) [n:127.0.0.1:40631_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1551255 INFO  (coreLoadExecutor-2140-thread-1-processing-n:127.0.0.1:40631_) [n:127.0.0.1:40631_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1551255 INFO  (coreLoadExecutor-2140-thread-1-processing-n:127.0.0.1:40631_) [n:127.0.0.1:40631_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 1551257 INFO  (coreLoadExecutor-2140-thread-1-processing-n:127.0.0.1:40631_) [n:127.0.0.1:40631_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1551258 INFO  (coreLoadExecutor-2140-thread-1-processing-n:127.0.0.1:40631_) [n:127.0.0.1:40631_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1551259 INFO  (coreLoadExecutor-2140-thread-1-processing-n:127.0.0.1:40631_) [n:127.0.0.1:40631_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1551260 INFO  (coreLoadExecutor-2140-thread-1-processing-n:127.0.0.1:40631_) [n:127.0.0.1:40631_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1551263 INFO  (coreLoadExecutor-2140-thread-1-processing-n:127.0.0.1:40631_) [n:127.0.0.1:40631_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/mbeans,standard,/update/csv,/update/json/docs,/admin/luke,/admin/segments,/get,/admin/system,/replication,/admin/properties,/config,/schema,/admin/plugins,/admin/logging,/update/json,/admin/threads,/admin/ping,/update,/admin/file
   [junit4]   2> 1551264 INFO  (coreLoadExecutor-2140-thread-1-processing-n:127.0.0.1:40631_) [n:127.0.0.1:40631_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 1551264 INFO  (coreLoadExecutor-2140-thread-1-processing-n:127.0.0.1:40631_) [n:127.0.0.1:40631_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1551265 INFO  (coreLoadExecutor-2140-thread-1-processing-n:127.0.0.1:40631_) [n:127.0.0.1:40631_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1551266 INFO  (coreLoadExecutor-2140-thread-1-processing-n:127.0.0.1:40631_) [n:127.0.0.1:40631_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 1551266 INFO  (coreLoadExecutor-2140-thread-1-processing-n:127.0.0.1:40631_) [n:127.0.0.1:40631_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 1551266 INFO  (coreLoadExecutor-2140-thread-1-processing-n:127.0.0.1:40631_) [n:127.0.0.1:40631_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=41, maxMergeAtOnceExplicit=21, maxMergedSegmentMB=47.0556640625, floorSegmentMB=0.26171875, forceMergeDeletesPctAllowed=24.89848619289661, segmentsPerTier=10.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 1551267 INFO  (coreLoadExecutor-2140-thread-1-processing-n:127.0.0.1:40631_) [n:127.0.0.1:40631_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@1b15f6e4 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@c5c1d1e),segFN=segments_1,generation=1}
   [junit4]   2> 1551267 INFO  (coreLoadExecutor-2140-thread-1-processing-n:127.0.0.1:40631_) [n:127.0.0.1:40631_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 1551267 INFO  (coreLoadExecutor-2140-thread-1-processing-n:127.0.0.1:40631_) [n:127.0.0.1:40631_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@74224ae1[collection1] main
   [junit4]   2> 1551267 INFO  (coreLoadExecutor-2140-thread-1-processing-n:127.0.0.1:40631_) [n:127.0.0.1:40631_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 1551269 INFO  (coreLoadExecutor-2140-thread-1-processing-n:127.0.0.1:40631_) [n:127.0.0.1:40631_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1551269 INFO  (coreLoadExecutor-2140-thread-1-processing-n:127.0.0.1:40631_) [n:127.0.0.1:40631_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 1551270 INFO  (coreLoadExecutor-2140-thread-1-processing-n:127.0.0.1:40631_) [n:127.0.0.1:40631_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1551270 INFO  (coreLoadExecutor-2140-thread-1-processing-n:127.0.0.1:40631_) [n:127.0.0.1:40631_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 1551270 INFO  (coreLoadExecutor-2140-thread-1-processing-n:127.0.0.1:40631_) [n:127.0.0.1:40631_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1551270 INFO  (coreLoadExecutor-2140-thread-1-processing-n:127.0.0.1:40631_) [n:127.0.0.1:40631_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 1551271 INFO  (coreLoadExecutor-2140-thread-1-processing-n:127.0.0.1:40631_) [n:127.0.0.1:40631_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1551271 INFO  (coreLoadExecutor-2140-thread-1-processing-n:127.0.0.1:40631_) [n:127.0.0.1:40631_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 1551271 INFO  (coreLoadExecutor-2140-thread-1-processing-n:127.0.0.1:40631_) [n:127.0.0.1:40631_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1551271 INFO  (searcherExecutor-2141-thread-1-processing-n:127.0.0.1:40631_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:40631_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@74224ae1[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1551274 INFO  (coreLoadExecutor-2140-thread-1-processing-n:127.0.0.1:40631_) [n:127.0.0.1:40631_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 1551274 INFO  (coreLoadExecutor-2140-thread-1-processing-n:127.0.0.1:40631_) [n:127.0.0.1:40631_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 1551274 INFO  (coreLoadExecutor-2140-thread-1-processing-n:127.0.0.1:40631_) [n:127.0.0.1:40631_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 1551274 INFO  (coreLoadExecutor-2140-thread-1-processing-n:127.0.0.1:40631_) [n:127.0.0.1:40631_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1507829798789971968
   [junit4]   2> 1551276 INFO  (coreLoadExecutor-2140-thread-1-processing-n:127.0.0.1:40631_) [n:127.0.0.1:40631_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 2 ms to seed version buckets with highest version 1507829798789971968
   [junit4]   2> 1551277 INFO  (coreLoadExecutor-2140-thread-1-processing-n:127.0.0.1:40631_) [n:127.0.0.1:40631_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 1551277 INFO  (coreLoadExecutor-2140-thread-1-processing-n:127.0.0.1:40631_) [n:127.0.0.1:40631_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 1551278 INFO  (coreZkRegister-2134-thread-1-processing-n:127.0.0.1:40631_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:40631_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:40631 collection:control_collection shard:shard1
   [junit4]   2> 1551279 INFO  (coreZkRegister-2134-thread-1-processing-n:127.0.0.1:40631_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:40631_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 1551283 INFO  (coreZkRegister-2134-thread-1-processing-n:127.0.0.1:40631_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:40631_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94239362304180228-core_node1-n_0000000000
   [junit4]   2> 1551285 INFO  (coreZkRegister-2134-thread-1-processing-n:127.0.0.1:40631_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:40631_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 1551287 INFO  (zkCallback-555-thread-1-processing-n:127.0.0.1:40631_) [n:127.0.0.1:40631_    ] o.a.s.c.DistributedQueue NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 1551287 INFO  (coreZkRegister-2134-thread-1-processing-n:127.0.0.1:40631_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:40631_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1551288 INFO  (coreZkRegister-2134-thread-1-processing-n:127.0.0.1:40631_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:40631_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1551288 INFO  (coreZkRegister-2134-thread-1-processing-n:127.0.0.1:40631_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:40631_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:40631/collection1/
   [junit4]   2> 1551288 INFO  (coreZkRegister-2134-thread-1-processing-n:127.0.0.1:40631_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:40631_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 1551288 INFO  (coreZkRegister-2134-thread-1-processing-n:127.0.0.1:40631_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:40631_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:40631/collection1/ has no replicas
   [junit4]   2> 1551289 INFO  (coreZkRegister-2134-thread-1-processing-n:127.0.0.1:40631_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:40631_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:40631/collection1/ shard1
   [junit4]   2> 1551289 INFO  (OverseerStateUpdate-94239362304180228-127.0.0.1:40631_-n_0000000000) [n:127.0.0.1:40631_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 1551289 INFO  (coreZkRegister-2134-thread-1-processing-n:127.0.0.1:40631_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:40631_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 1551293 INFO  (zkCallback-555-thread-1-processing-n:127.0.0.1:40631_) [n:127.0.0.1:40631_    ] o.a.s.c.DistributedQueue NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 1551294 INFO  (OverseerStateUpdate-94239362304180228-127.0.0.1:40631_-n_0000000000) [n:127.0.0.1:40631_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:40631",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 1
   [junit4]   2> 1551397 INFO  (zkCallback-555-thread-1-processing-n:127.0.0.1:40631_) [n:127.0.0.1:40631_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1551401 INFO  (zkCallback-555-thread-1-processing-n:127.0.0.1:40631_) [n:127.0.0.1:40631_    ] o.a.s.c.c.ZkStateReader Updated cluster state version to 2
   [junit4]   2> 1551444 INFO  (coreZkRegister-2134-thread-1-processing-n:127.0.0.1:40631_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:40631_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:40631/collection1/ and leader is http://127.0.0.1:40631/collection1/
   [junit4]   2> 1551444 INFO  (coreZkRegister-2134-thread-1-processing-n:127.0.0.1:40631_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:40631_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:40631
   [junit4]   2> 1551444 INFO  (coreZkRegister-2134-thread-1-processing-n:127.0.0.1:40631_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:40631_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1551445 INFO  (coreZkRegister-2134-thread-1-processing-n:127.0.0.1:40631_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:40631_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 1551445 INFO  (coreZkRegister-2134-thread-1-processing-n:127.0.0.1:40631_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:40631_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1551446 INFO  (zkCallback-555-thread-1-processing-n:127.0.0.1:40631_) [n:127.0.0.1:40631_    ] o.a.s.c.DistributedQueue NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 1551447 INFO  (OverseerStateUpdate-94239362304180228-127.0.0.1:40631_-n_0000000000) [n:127.0.0.1:40631_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:40631",
   [junit4]   2>   "node_name":"127.0.0.1:40631_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"} current state version: 2
   [junit4]   2> 1551448 INFO  (OverseerStateUpdate-94239362304180228-127.0.0.1:40631_-n_0000000000) [n:127.0.0.1:40631_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:40631",
   [junit4]   2>   "node_name":"127.0.0.1:40631_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 1551545 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1551547 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1551548 INFO  (zkCallback-557-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@448ab6ba name:ZooKeeperConnection Watcher:127.0.0.1:36981/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1551548 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1551548 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1551549 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1551552 INFO  (zkCallback-557-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1551553 INFO  (zkCallback-555-thread-1-processing-n:127.0.0.1:40631_) [n:127.0.0.1:40631_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1551553 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 1551562 INFO  (zkCallback-557-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated cluster state version to 3
   [junit4]   2> 1551562 INFO  (zkCallback-555-thread-1-processing-n:127.0.0.1:40631_) [n:127.0.0.1:40631_    ] o.a.s.c.c.ZkStateReader Updated cluster state version to 3
   [junit4]   2> 1551937 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_7B9D114E4CC14E06-001/shard-1-001/cores/collection1
   [junit4]   2> 1551939 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_7B9D114E4CC14E06-001/shard-1-001
   [junit4]   2> 1551940 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [    ] o.e.j.s.Server jetty-9.2.11.v20150529
   [junit4]   2> 1551943 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@691a0e94{/,null,AVAILABLE}
   [junit4]   2> 1551943 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [    ] o.e.j.s.ServerConnector Started ServerConnector@62168aca{HTTP/1.1}{127.0.0.1:?????}
   [junit4]   2> 1551943 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [    ] o.e.j.s.Server Started @???????ms
   [junit4]   2> 1551944 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_7B9D114E4CC14E06-001/tempDir-001/jetty1, solrconfig=solrconfig.xml, hostContext=/, hostPort=43373, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_7B9D114E4CC14E06-001/shard-1-001/cores}
   [junit4]   2> 1551944 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@4e0e2f2a
   [junit4]   2> 1551944 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_7B9D114E4CC14E06-001/shard-1-001/'
   [junit4]   2> 1551967 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1551976 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1551977 INFO  (zkCallback-558-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@524b5324 name:ZooKeeperConnection Watcher:127.0.0.1:36981/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1551977 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1551978 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1551979 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1551979 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_7B9D114E4CC14E06-001/shard-1-001/solr.xml
   [junit4]   2> 1551986 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_7B9D114E4CC14E06-001/shard-1-001/cores
   [junit4]   2> 1551987 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [    ] o.a.s.c.CoreContainer New CoreContainer 1430251618
   [junit4]   2> 1551987 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_7B9D114E4CC14E06-001/shard-1-001/]
   [junit4]   2> 1551987 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_7B9D114E4CC14E06-001/shard-1-001/lib
   [junit4]   2> 1551987 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_7B9D114E4CC14E06-001/shard-1-001/lib).
   [junit4]   2> 1551997 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1551998 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1551999 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1551999 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1551999 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:36981/solr
   [junit4]   2> 1551999 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1551999 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1552002 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1552003 INFO  (zkCallback-560-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6b9da0fd name:ZooKeeperConnection Watcher:127.0.0.1:36981 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1552003 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1552004 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1552007 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [n:127.0.0.1:43373_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1552009 INFO  (zkCallback-561-thread-1-processing-n:127.0.0.1:43373_) [n:127.0.0.1:43373_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@74b71a29 name:ZooKeeperConnection Watcher:127.0.0.1:36981/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1552009 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [n:127.0.0.1:43373_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1552016 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [n:127.0.0.1:43373_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1553019 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [n:127.0.0.1:43373_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:43373_
   [junit4]   2> 1553020 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [n:127.0.0.1:43373_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:43373_
   [junit4]   2> 1553027 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [n:127.0.0.1:43373_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1553028 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [n:127.0.0.1:43373_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94239362304180232-127.0.0.1:43373_-n_0000000001
   [junit4]   2> 1553029 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [n:127.0.0.1:43373_    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/94239362304180228-127.0.0.1:40631_-n_0000000000 to know if I could be the leader
   [junit4]   2> 1553029 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [n:127.0.0.1:43373_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1553030 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [n:127.0.0.1:43373_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1553031 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [n:127.0.0.1:43373_    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_7B9D114E4CC14E06-001/shard-1-001/cores
   [junit4]   2> 1553031 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [n:127.0.0.1:43373_    ] o.a.s.c.SolrCore Created CoreDescriptor: {name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, loadOnStartup=true, configSetProperties=configsetprops.json, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_7B9D114E4CC14E06-001/shard-1-001/cores/collection1, collection=collection1, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_7B9D114E4CC14E06-001/shard-1-001/cores/collection1/, coreNodeName=, dataDir=data/, shard=}
   [junit4]   2> 1553032 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [n:127.0.0.1:43373_    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_7B9D114E4CC14E06-001/shard-1-001/cores/collection1/
   [junit4]   2> 1553032 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [n:127.0.0.1:43373_    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 1553037 INFO  (coreLoadExecutor-2151-thread-1-processing-n:127.0.0.1:43373_) [n:127.0.0.1:43373_ c:collection1   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 1553037 INFO  (coreLoadExecutor-2151-thread-1-processing-n:127.0.0.1:43373_) [n:127.0.0.1:43373_ c:collection1   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1553038 INFO  (coreLoadExecutor-2151-thread-1-processing-n:127.0.0.1:43373_) [n:127.0.0.1:43373_ c:collection1   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 1553038 INFO  (zkCallback-555-thread-1-processing-n:127.0.0.1:40631_) [n:127.0.0.1:40631_    ] o.a.s.c.DistributedQueue NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 1553039 INFO  (OverseerStateUpdate-94239362304180228-127.0.0.1:40631_-n_0000000000) [n:127.0.0.1:40631_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:43373",
   [junit4]   2>   "node_name":"127.0.0.1:43373_",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "operation":"state"} current state version: 3
   [junit4]   2> 1553039 INFO  (OverseerStateUpdate-94239362304180228-127.0.0.1:40631_-n_0000000000) [n:127.0.0.1:40631_    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:43373",
   [junit4]   2>   "node_name":"127.0.0.1:43373_",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 1553039 INFO  (OverseerStateUpdate-94239362304180228-127.0.0.1:40631_-n_0000000000) [n:127.0.0.1:40631_    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 1553039 INFO  (OverseerStateUpdate-94239362304180228-127.0.0.1:40631_-n_0000000000) [n:127.0.0.1:40631_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2
   [junit4]   2> 1553049 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [n:127.0.0.1:43373_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1
   [junit4]   2> 1553050 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7B9D114E4CC14E06]) [n:127.0.0.1:43373_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1553142 INFO  (zkCallback-561-thread-1-processing-n:127.0.0.1:43373_) [n:127.0.0.1:43373_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1553142 INFO  (zkCallback-557-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1553143 INFO  (zkCallback-555-thread-1-processing-n:127.0.0.1:40631_) [n:127.0.0.1:40631_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1553145 INFO  (zkCallback-557-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated cluster state version to 4
   [junit4]   2> 1553145 INFO  (zkCallback-561-thread-1-processing-n:127.0.0.1:43373_) [n:127.0.0.1:43373_    ] o.a.s.c.c.ZkStateReader Updated cluster state version to 4
   [junit4]   2> 1553146 INFO  (zkCallback-555-thread-1-processing-n:127.0.0.1:40631_) [n:127.0.0.1:40631_    ] o.a.s.c.c.ZkStateReader Updated cluster state version to 4
   [junit4]   2> 1554038 INFO  (coreLoadExecutor-2151-thread-1-processing-n:127.0.0.1:43373_) [n:127.0.0.1:43373_ c:collection1   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1554038 INFO  (coreLoadExecutor-2151-thread-1-processing-n:127.0.0.1:43373_) [n:127.0.0.1:43373_ c:collection1   x:collection1] o.a.s.c.ZkController Check for collection zkNode:collection1
   [junit4]   2> 1554039 INFO  (coreLoadExecutor-2151-thread-1-processing-n:127.0.0.1:43373_) [n:127.0.0.1:43373_ c:collection1   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 1554039 INFO  (coreLoadExecutor-2151-thread-1-processing-n:127.0.0.1:43373_) [n:127.0.0.1:43373_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/collection1
   [junit4]   2> 1554039 INFO  (coreLoadExecutor-2151-thread-1-processing-n:127.0.0.1:43373_) [n:127.0.0.1:43373_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1554039 INFO  (coreLoadExecutor-2151-thread-1-processing-n:127.0.0.1:43373_) [n:127.0.0.1:43373_ c:collection1   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_7B9D114E4CC14E06-001/shard-1-001/cores/collection1/'
   [junit4]   2> 1554054 INFO  (coreLoadExecutor-2151-thread-1-processing-n:127.0.0.1:43373_) [n:127.0.0.1:43373_ c:collection1   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 1554059 INFO  (coreLoadExecutor-2151-thread-1-processing-n:127.0.0.1:43373_) [n:127.0.0.1:43373_ c:collection1   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 1554063 INFO  (coreLoadExecutor-2151-thread-1-processing-n:127.0.0.1:43373_) [n:127.0.0.1:43373_ c:collection1   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.0.0
   [junit4]   2> 1554070 INFO  (coreLoadExecutor-2151-thread-1-processing-n:127.0.0.1:43373_) [n:127.0.0.1:43373_ c:collection1   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1554071 INFO  (coreLoadExecutor-2151-thread-1-processing-n:127.0.0.1:43373_) [n:127.0.0.1:43373_ c:collection1   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 1554076 INFO  (coreLoadExecutor-2151-thread-1-processing-n:127.0.0.1:43373_) [n:127.0.0.1:43373_ c:collection1   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1554186 INFO  (coreLoadExecutor-2151-thread-1-processing-n:127.0.0.1:43373_) [n:127.0.0.1:43373_ c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 1554193 INFO  (coreLoadExecutor-2151-thread-1-processing-n:127.0.0.1:43373_) [n:127.0.0.1:43373_ c:collection1   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 1554194 INFO  (coreLoadExecutor-2151-thread-1-processing-n:127.0.0.1:43373_) [n:127.0.0.1:43373_ c:collection1   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 1554202 INFO  (coreLoadExecutor-2151-thread-1-processing-n:127.0.0.1:43373_) [n:127.0.0.1:43373_ c:collection1   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1554204 INFO  (coreLoadExecutor-2151-thread-1-processing-n:127.0.0.1:43373_) [n:127.0.0.1:43373_ c:collection1   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1554206 INFO  (coreLoadExecutor-2151-thread-1-processing-n:127.0.0.1:43373_) [n:127.0.0.1:43373_ c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1554207 WARN  (coreLoadExecutor-2151-thread-1-processing-n:127.0.0.1:43373_) [n:127.0.0.1:43373_ c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 1554207 WARN  (coreLoadExecutor-2151-thread-1-processing-n:127.0.0.1:43373_) [n:127.0.0.1:43373_ c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 1554207 INFO  (coreLoadExecutor-2151-thread-1-processing-n:127.0.0.1:43373_) [n:127.0.0.1:43373_ c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1554208 WARN  (coreLoadExecutor-2151-thread-1-processing-n:127.0.0.1:43373_) [n:127.0.0.1:43373_ c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 1554208 WARN  (coreLoadExecutor-2151-thread-1-processing-n:127.0.0.1:43373_) [n:127.0.0.1:43373_ c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 1554209 INFO  (coreLoadExecutor-2151-thread-1-processing-n:127.0.0.1:43373_) [n:127.0.0.1:43373_ c:collection1   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1
   [junit4]   2> 1554209 INFO  (coreLoadExecutor-2151-thread-1-processing-n:127.0.0.1:43373_) [n:127.0.0.1:43373_ c:collection1   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection collection1
   [junit4]   2> 1554209 INFO  (coreLoadExecutor-2151-thread-1-processing-n:127.0.0.1:43373_) [n:127.0.0.1:43373_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 1554209 INFO  (coreLoadExecutor-2151-thread-1-processing-n:127.0.0.1:43373_) [n:127.0.0.1:43373_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_7B9D114E4CC14E06-001/shard-1-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 1554210 INFO  (coreLoadExecutor-2151-thread-1-processing-n:127.0.0.1:43373_) [n:127.0.0.1:43373_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@699fe850
   [junit4]   2> 1554210 INFO  (coreLoadExecutor-2151-thread-1-processing-n:127.0.0.1:43373_) [n:127.0.0.1:43373_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_7B9D114E4CC14E06-001/shard-1-001/cores/collection1/data
   [junit4]   2> 1554210 INFO  (coreLoadExecutor-2151-thread-1-processing-n:127.0.0.1:43373_) [n:127.0.0.1:43373_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_7B9D114E4CC14E06-001/shard-1-001/cores/collection1/data/index/
   [junit4]   2> 1554210 WARN  (coreLoadExecutor-2151-thread-1-processing-n:127.0.0.1:43373_) [n:127.0.0.1:43373_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_7B9D114E4CC14E06-001/shard-1-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 1554211 INFO  (coreLoadExecutor-2151-thread-1-processing-n:127.0.0.1:43373_) [n:127.0.0.1:43373_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_7B9D114E4CC14E06-001/shard-1-001/cores/collection1/data/index
   [junit4]   2> 1554211 INFO  (coreLoadExecutor-2151-thread-1-processing-n:127.0.0.1:43373_) [n:127.0.0.1:43373_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=32, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.5004314197358152]
   [junit4]   2

[...truncated too long message...]

M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/apacheds-mavibot-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/apacheds-protocol-kerberos-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/apacheds-protocol-ldap-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/apacheds-protocol-shared-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/apacheds-xdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/api-all-1.0.0-M20.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/bcprov-jdk15-1.45.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/cglib-nodep-2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/commons-collections-3.2.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/commons-math3-3.4.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/easymock-3.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/ehcache-core-2.4.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/hadoop-common-2.6.0-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/hadoop-hdfs-2.6.0-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/hadoop-minikdc-2.6.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/jackson-annotations-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/jackson-databind-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/jcl-over-slf4j-1.7.7.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/jersey-core-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/jersey-server-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/jetty-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/jetty-util-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/mina-core-2.0.0-M5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/objenesis-1.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/presto-parser-0.108.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/slice-0.10.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/build/analysis/icu/lucene-analyzers-icu-6.0.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/contrib/solr-analysis-extras/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/contrib/analysis-extras/lib/icu4j-54.1.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-launcher.jar:/x1/jenkins/.ant/lib/ivy-2.3.0.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-antlr.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-regexp.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-resolver.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-net.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jmf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-oro.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-xalan2.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-logging.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jsch.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit4.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-javamail.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jai.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jdepend.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bsf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-swing.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-netrexx.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bcel.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-testutil.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-log4j.jar:/x1/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.8/lib/tools.jar:/x1/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.1.13.jar com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/temp/junit4-J2-20150727_020318_007.events @/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/temp/junit4-J2-20150727_020318_007.suites
   [junit4] ERROR: JVM J2 ended with an exception: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1504)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:133)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:964)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:961)
   [junit4] 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4] 	at java.lang.Thread.run(Thread.java:745)

BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:724: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:661: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build.xml:230: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/common-build.xml:517: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/common-build.xml:1429: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/common-build.xml:984: At least one slave process threw an exception, first: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.

Total time: 393 minutes 59 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
No prior successful build to compare, so performing full copy of artifacts
ERROR: Failed to archive artifacts: **/*.events,heapdumps/**,**/hs_err_pid*,README.txt
java.io.IOException: Failed to extract /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/transfer of 4 files
	at hudson.FilePath.readFromTar(FilePath.java:2101)
	at hudson.FilePath.copyRecursiveTo(FilePath.java:2013)
	at jenkins.model.StandardArtifactManager.archive(StandardArtifactManager.java:61)
	at com.cloudbees.jenkins.plugins.jsync.archiver.JSyncArtifactManager.remoteSync(JSyncArtifactManager.java:107)
	at com.cloudbees.jenkins.plugins.jsync.archiver.JSyncArtifactManager.archive(JSyncArtifactManager.java:68)
	at hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:140)
	at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
	at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:756)
	at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:720)
	at hudson.model.Build$BuildExecution.post2(Build.java:182)
	at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:669)
	at hudson.model.Run.execute(Run.java:1731)
	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
	at hudson.model.ResourceController.execute(ResourceController.java:88)
	at hudson.model.Executor.run(Executor.java:232)
Caused by: java.io.IOException: unexpected EOF with 1536 bytes unread
	at hudson.org.apache.tools.tar.TarInputStream.read(TarInputStream.java:349)
	at java.io.FilterInputStream.read(FilterInputStream.java:107)
	at org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:1792)
	at org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:1769)
	at org.apache.commons.io.IOUtils.copy(IOUtils.java:1744)
	at hudson.util.IOUtils.copy(IOUtils.java:40)
	at hudson.FilePath.readFromTar(FilePath.java:2091)
	... 14 more
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-NightlyTests-trunk - Build # 747 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-trunk/747/

All tests passed

Build Log:
[...truncated 12155 lines...]
   [junit4] JVM J2: stderr was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/temp/junit4-J2-20150726_093202_200.syserr
   [junit4] >>> JVM J2: stderr (verbatim) ----
   [junit4] WARN: Unhandled exception in event serialization. -> com.carrotsearch.ant.tasks.junit4.dependencies.com.google.gson.JsonIOException: java.io.IOException: No space left on device
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.dependencies.com.google.gson.Gson.toJson(Gson.java:514)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.events.Serializer.serialize(Serializer.java:87)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain$4.write(SlaveMain.java:410)
   [junit4] 	at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
   [junit4] 	at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
   [junit4] 	at java.io.PrintStream.flush(PrintStream.java:338)
   [junit4] 	at java.io.FilterOutputStream.flush(FilterOutputStream.java:140)
   [junit4] 	at java.io.PrintStream.write(PrintStream.java:482)
   [junit4] 	at sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:221)
   [junit4] 	at sun.nio.cs.StreamEncoder.implFlushBuffer(StreamEncoder.java:291)
   [junit4] 	at sun.nio.cs.StreamEncoder.implFlush(StreamEncoder.java:295)
   [junit4] 	at sun.nio.cs.StreamEncoder.flush(StreamEncoder.java:141)
   [junit4] 	at java.io.OutputStreamWriter.flush(OutputStreamWriter.java:229)
   [junit4] 	at org.apache.log4j.helpers.QuietWriter.flush(QuietWriter.java:59)
   [junit4] 	at org.apache.log4j.WriterAppender.subAppend(WriterAppender.java:324)
   [junit4] 	at org.apache.log4j.WriterAppender.append(WriterAppender.java:162)
   [junit4] 	at org.apache.log4j.AppenderSkeleton.doAppend(AppenderSkeleton.java:251)
   [junit4] 	at org.apache.log4j.helpers.AppenderAttachableImpl.appendLoopOnAppenders(AppenderAttachableImpl.java:66)
   [junit4] 	at org.apache.log4j.Category.callAppenders(Category.java:206)
   [junit4] 	at org.apache.log4j.Category.forcedLog(Category.java:391)
   [junit4] 	at org.apache.log4j.Category.log(Category.java:856)
   [junit4] 	at org.slf4j.impl.Log4jLoggerAdapter.log(Log4jLoggerAdapter.java:601)
   [junit4] 	at org.apache.commons.logging.impl.SLF4JLocationAwareLog.warn(SLF4JLocationAwareLog.java:199)
   [junit4] 	at org.apache.hadoop.hdfs.server.datanode.DataXceiverServer.run(DataXceiverServer.java:158)
   [junit4] 	at java.lang.Thread.run(Thread.java:745)
   [junit4] Caused by: java.io.IOException: No space left on device
   [junit4] 	at java.io.FileOutputStream.writeBytes(Native Method)
   [junit4] 	at java.io.FileOutputStream.write(FileOutputStream.java:326)
   [junit4] 	at java.io.BufferedOutputStream.write(BufferedOutputStream.java:122)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.slave.EventsOutputStream.write(EventsOutputStream.java:28)
   [junit4] 	at sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:221)
   [junit4] 	at sun.nio.cs.StreamEncoder.implWrite(StreamEncoder.java:282)
   [junit4] 	at sun.nio.cs.StreamEncoder.write(StreamEncoder.java:125)
   [junit4] 	at sun.nio.cs.StreamEncoder.write(StreamEncoder.java:113)
   [junit4] 	at java.io.OutputStreamWriter.write(OutputStreamWriter.java:194)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.dependencies.com.google.gson.stream.JsonWriter.string(JsonWriter.java:535)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.dependencies.com.google.gson.stream.JsonWriter.value(JsonWriter.java:364)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.dependencies.com.google.gson.internal.bind.TypeAdapters$22.write(TypeAdapters.java:626)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.dependencies.com.google.gson.internal.bind.TypeAdapters$22.write(TypeAdapters.java:578)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.dependencies.com.google.gson.internal.Streams.write(Streams.java:67)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.dependencies.com.google.gson.GsonToMiniGsonTypeAdapterFactory$3.write(GsonToMiniGsonTypeAdapterFactory.java:98)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.dependencies.com.google.gson.internal.bind.TypeAdapterRuntimeTypeWrapper.write(TypeAdapterRuntimeTypeWrapper.java:66)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.dependencies.com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.write(ReflectiveTypeAdapterFactory.java:82)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.dependencies.com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.write(ReflectiveTypeAdapterFactory.java:194)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.dependencies.com.google.gson.Gson.toJson(Gson.java:512)
   [junit4] 	... 24 more
   [junit4] <<< JVM J2: EOF ----

[...truncated 8 lines...]
   [junit4] ERROR: JVM J2 ended with an exception, command line: /x1/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.8/jre/bin/java -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/heapdumps -ea -esa -Dtests.prefix=tests -Dtests.seed=D55884826BDE4631 -Xmx512M -Dtests.iters= -Dtests.verbose=false -Dtests.infostream=false -Dtests.codec=random -Dtests.postingsformat=random -Dtests.docvaluesformat=random -Dtests.locale=random -Dtests.timezone=random -Dtests.directory=random -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.luceneMatchVersion=6.0.0 -Dtests.cleanthreads=perClass -Djava.util.logging.config.file=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/tools/junit4/logging.properties -Dtests.nightly=true -Dtests.weekly=false -Dtests.monster=false -Dtests.slow=true -Dtests.asserts=true -Dtests.multiplier=2 -DtempDir=./temp -Djava.io.tmpdir=./temp -Djunit4.tempDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/temp -Dcommon.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene -Dclover.db.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/build/clover/db -Djava.security.policy=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/tools/junit4/solr-tests.policy -Dtests.LUCENE_VERSION=6.0.0 -Djetty.testMode=1 -Djetty.insecurerandom=1 -Dsolr.directoryFactory=org.apache.solr.core.MockDirectoryFactory -Djava.awt.headless=true -Djdk.map.althashing.threshold=0 -Dtests.leaveTemporary=false -Dtests.filterstacks=true -Djava.security.manager=org.apache.lucene.util.TestSecurityManager -classpath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/classes/test:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-test-framework/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/test-framework/lib/junit4-ant-2.1.13.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/build/test-framework/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/build/codecs/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-solrj/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/build/analysis/common/lucene-analyzers-common-6.0.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/build/analysis/kuromoji/lucene-analyzers-kuromoji-6.0.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/build/analysis/phonetic/lucene-analyzers-phonetic-6.0.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/build/codecs/lucene-codecs-6.0.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/build/backward-codecs/lucene-backward-codecs-6.0.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/build/highlighter/lucene-highlighter-6.0.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/build/memory/lucene-memory-6.0.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/build/misc/lucene-misc-6.0.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/build/spatial/lucene-spatial-6.0.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/build/expressions/lucene-expressions-6.0.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/build/suggest/lucene-suggest-6.0.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/build/grouping/lucene-grouping-6.0.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/build/queries/lucene-queries-6.0.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/build/queryparser/lucene-queryparser-6.0.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/build/join/lucene-join-6.0.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/build/sandbox/lucene-sandbox-6.0.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/lib/antlr-runtime-3.5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/lib/antlr4-runtime-4.5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/lib/asm-4.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/lib/asm-commons-4.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/lib/caffeine-1.0.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/lib/commons-cli-1.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/lib/commons-codec-1.10.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/lib/commons-collections-3.2.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/lib/commons-configuration-1.6.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/lib/commons-fileupload-1.2.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/lib/commons-lang-2.6.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/lib/dom4j-1.6.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/lib/guava-14.0.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/lib/hadoop-annotations-2.6.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/lib/hadoop-auth-2.6.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/lib/hadoop-common-2.6.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/lib/hadoop-hdfs-2.6.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/lib/hppc-0.5.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/lib/htrace-core-3.0.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/lib/jackson-core-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/lib/jackson-dataformat-smile-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/lib/joda-time-2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/lib/log4j-1.2.17.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/lib/org.restlet-2.3.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/lib/org.restlet.ext.servlet-2.3.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/lib/presto-parser-0.108.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/lib/protobuf-java-2.5.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/lib/slf4j-log4j12-1.7.7.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/lib/slice-0.10.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/lib/spatial4j-0.4.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/lib/t-digest-3.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/solrj/lib/commons-io-2.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/solrj/lib/httpclient-4.4.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/solrj/lib/httpcore-4.4.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/solrj/lib/httpmime-4.4.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/solrj/lib/noggit-0.6.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/solrj/lib/slf4j-api-1.7.7.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/solrj/lib/stax2-api-3.1.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/solrj/lib/woodstox-core-asl-4.4.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/solrj/lib/zookeeper-3.4.6.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/server/lib/javax.servlet-api-3.1.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/server/lib/jetty-continuation-9.2.11.v20150529.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/server/lib/jetty-deploy-9.2.11.v20150529.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/server/lib/jetty-http-9.2.11.v20150529.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/server/lib/jetty-io-9.2.11.v20150529.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/server/lib/jetty-jmx-9.2.11.v20150529.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/server/lib/jetty-rewrite-9.2.11.v20150529.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/server/lib/jetty-security-9.2.11.v20150529.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/server/lib/jetty-server-9.2.11.v20150529.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/server/lib/jetty-servlet-9.2.11.v20150529.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/server/lib/jetty-servlets-9.2.11.v20150529.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/server/lib/jetty-util-9.2.11.v20150529.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/server/lib/jetty-webapp-9.2.11.v20150529.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/server/lib/jetty-xml-9.2.11.v20150529.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/example/example-DIH/solr/db/lib/derby-10.9.1.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/example/example-DIH/solr/db/lib/hsqldb-1.8.0.10.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/build/core/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/test-framework/lib/junit-4.10.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/test-framework/lib/randomizedtesting-runner-2.1.13.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/antlr-2.7.7.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/antlr4-runtime-4.5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/apacheds-core-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/apacheds-core-api-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/apacheds-core-avl-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/apacheds-core-shared-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/apacheds-i18n-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/apacheds-interceptor-kerberos-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/apacheds-interceptors-admin-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/apacheds-interceptors-authn-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/apacheds-interceptors-authz-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/apacheds-interceptors-changelog-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/apacheds-interceptors-collective-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/apacheds-interceptors-event-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/apacheds-interceptors-exception-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/apacheds-interceptors-journal-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/apacheds-interceptors-normalization-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/apacheds-interceptors-operational-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/apacheds-interceptors-referral-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/apacheds-interceptors-schema-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/apacheds-interceptors-subtree-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/apacheds-interceptors-trigger-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/apacheds-jdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/apacheds-jdbm1-2.0.0-M2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/apacheds-kerberos-codec-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/apacheds-ldif-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/apacheds-mavibot-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/apacheds-protocol-kerberos-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/apacheds-protocol-ldap-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/apacheds-protocol-shared-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/apacheds-xdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/api-all-1.0.0-M20.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/bcprov-jdk15-1.45.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/cglib-nodep-2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/commons-collections-3.2.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/commons-math3-3.4.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/easymock-3.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/ehcache-core-2.4.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/hadoop-common-2.6.0-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/hadoop-hdfs-2.6.0-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/hadoop-minikdc-2.6.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/jackson-annotations-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/jackson-databind-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/jcl-over-slf4j-1.7.7.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/jersey-core-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/jersey-server-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/jetty-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/jetty-util-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/mina-core-2.0.0-M5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/objenesis-1.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/presto-parser-0.108.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/slice-0.10.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/build/analysis/icu/lucene-analyzers-icu-6.0.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/contrib/solr-analysis-extras/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/contrib/analysis-extras/lib/icu4j-54.1.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-launcher.jar:/x1/jenkins/.ant/lib/ivy-2.3.0.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-antlr.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-regexp.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-resolver.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-net.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jmf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-oro.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-xalan2.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-logging.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jsch.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit4.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-javamail.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jai.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jdepend.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bsf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-swing.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-netrexx.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bcel.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-testutil.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-log4j.jar:/x1/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.8/lib/tools.jar:/x1/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.1.13.jar com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/temp/junit4-J2-20150726_093202_200.events @/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/temp/junit4-J2-20150726_093202_200.suites
   [junit4] ERROR: JVM J2 ended with an exception: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1504)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:133)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:964)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:961)
   [junit4] 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4] 	at java.lang.Thread.run(Thread.java:745)

BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:724: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:661: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build.xml:230: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/common-build.xml:517: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/common-build.xml:1429: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/common-build.xml:984: At least one slave process threw an exception, first: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.

Total time: 397 minutes 25 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
No prior successful build to compare, so performing full copy of artifacts
ERROR: Failed to archive artifacts: **/*.events,heapdumps/**,**/hs_err_pid*,README.txt
java.io.IOException: Failed to extract /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/transfer of 4 files
	at hudson.FilePath.readFromTar(FilePath.java:2101)
	at hudson.FilePath.copyRecursiveTo(FilePath.java:2013)
	at jenkins.model.StandardArtifactManager.archive(StandardArtifactManager.java:61)
	at com.cloudbees.jenkins.plugins.jsync.archiver.JSyncArtifactManager.remoteSync(JSyncArtifactManager.java:107)
	at com.cloudbees.jenkins.plugins.jsync.archiver.JSyncArtifactManager.archive(JSyncArtifactManager.java:68)
	at hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:140)
	at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
	at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:756)
	at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:720)
	at hudson.model.Build$BuildExecution.post2(Build.java:182)
	at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:669)
	at hudson.model.Run.execute(Run.java:1731)
	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
	at hudson.model.ResourceController.execute(ResourceController.java:88)
	at hudson.model.Executor.run(Executor.java:232)
Caused by: java.io.IOException: unexpected EOF with 512 bytes unread
	at hudson.org.apache.tools.tar.TarInputStream.read(TarInputStream.java:349)
	at java.io.FilterInputStream.read(FilterInputStream.java:107)
	at org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:1792)
	at org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:1769)
	at org.apache.commons.io.IOUtils.copy(IOUtils.java:1744)
	at hudson.util.IOUtils.copy(IOUtils.java:40)
	at hudson.FilePath.readFromTar(FilePath.java:2091)
	... 14 more
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-NightlyTests-trunk - Build # 746 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-trunk/746/

2 tests failed.
REGRESSION:  org.apache.solr.handler.TestReplicationHandler.doTestStressReplication

Error Message:
[replication.properties, index.20150725130634765, index.properties, index.20150725130630315] expected:<1> but was:<2>

Stack Trace:
java.lang.AssertionError: [replication.properties, index.20150725130634765, index.properties, index.20150725130630315] expected:<1> but was:<2>
	at __randomizedtesting.SeedInfo.seed([EC5FBD1647494C2F:37F4BDD04261259C]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.failNotEquals(Assert.java:647)
	at org.junit.Assert.assertEquals(Assert.java:128)
	at org.junit.Assert.assertEquals(Assert.java:472)
	at org.apache.solr.handler.TestReplicationHandler.checkForSingleIndex(TestReplicationHandler.java:818)
	at org.apache.solr.handler.TestReplicationHandler.doTestStressReplication(TestReplicationHandler.java:785)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:497)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=4348, name=collection1, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=4348, name=collection1, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:58453: Could not find collection : awholynewstresscollection_collection1_0
	at __randomizedtesting.SeedInfo.seed([EC5FBD1647494C2F]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:560)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:234)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:226)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:376)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:328)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1086)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:856)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:799)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:894)




Build Log:
[...truncated 10411 lines...]
   [junit4] Suite: org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_EC5FBD1647494C2F-001/init-core-data-001
   [junit4]   2> 322553 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[EC5FBD1647494C2F]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false)
   [junit4]   2> 322553 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[EC5FBD1647494C2F]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   1> Formatting using clusterid: testClusterID
   [junit4]   2> 322598 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[EC5FBD1647494C2F]-worker) [    ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
   [junit4]   2> 322612 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[EC5FBD1647494C2F]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 322616 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[EC5FBD1647494C2F]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 322633 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[EC5FBD1647494C2F]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/hdfs to ./temp/Jetty_localhost_34374_hdfs____.vgqltz/webapp
   [junit4]   2> 322757 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[EC5FBD1647494C2F]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 323188 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[EC5FBD1647494C2F]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:34374
   [junit4]   2> 323265 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[EC5FBD1647494C2F]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 323267 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[EC5FBD1647494C2F]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 323288 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[EC5FBD1647494C2F]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_36027_datanode____vgzk7t/webapp
   [junit4]   2> 323394 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[EC5FBD1647494C2F]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 323785 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[EC5FBD1647494C2F]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:36027
   [junit4]   2> 323847 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[EC5FBD1647494C2F]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 323851 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[EC5FBD1647494C2F]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 323875 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[EC5FBD1647494C2F]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_47758_datanode____.n3cste/webapp
   [junit4]   2> 323932 INFO  (IPC Server handler 4 on 41535) [    ] BlockStateChange BLOCK* processReport: from storage DS-3bb541e2-abde-4324-9705-e51f67e84540 node DatanodeRegistration(127.0.0.1, datanodeUuid=dd7f15c4-e6f9-4d65-adc7-a977ac02e0d8, infoPort=36027, ipcPort=49972, storageInfo=lv=-56;cid=testClusterID;nsid=257256401;c=0), blocks: 0, hasStaleStorages: true, processing time: 0 msecs
   [junit4]   2> 323933 INFO  (IPC Server handler 4 on 41535) [    ] BlockStateChange BLOCK* processReport: from storage DS-2c7c6212-04cf-4343-a4ed-f571c045f969 node DatanodeRegistration(127.0.0.1, datanodeUuid=dd7f15c4-e6f9-4d65-adc7-a977ac02e0d8, infoPort=36027, ipcPort=49972, storageInfo=lv=-56;cid=testClusterID;nsid=257256401;c=0), blocks: 0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 324002 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[EC5FBD1647494C2F]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 325470 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[EC5FBD1647494C2F]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:47758
   [junit4]   2> 325583 INFO  (IPC Server handler 8 on 41535) [    ] BlockStateChange BLOCK* processReport: from storage DS-7d9016e2-4e03-40cf-a44a-a294e530f1e7 node DatanodeRegistration(127.0.0.1, datanodeUuid=7b0dac37-5002-4781-a663-70d0ed4562f0, infoPort=47758, ipcPort=48403, storageInfo=lv=-56;cid=testClusterID;nsid=257256401;c=0), blocks: 0, hasStaleStorages: true, processing time: 0 msecs
   [junit4]   2> 325585 INFO  (IPC Server handler 8 on 41535) [    ] BlockStateChange BLOCK* processReport: from storage DS-09b68d51-a376-4be0-9ee6-b491bd109ec5 node DatanodeRegistration(127.0.0.1, datanodeUuid=7b0dac37-5002-4781-a663-70d0ed4562f0, infoPort=47758, ipcPort=48403, storageInfo=lv=-56;cid=testClusterID;nsid=257256401;c=0), blocks: 0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 325646 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 325647 INFO  (Thread-1875) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 325647 INFO  (Thread-1875) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 325747 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.ZkTestServer start zk server on port:54141
   [junit4]   2> 325747 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 325748 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 325757 INFO  (zkCallback-337-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@170957b1 name:ZooKeeperConnection Watcher:127.0.0.1:54141 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 325757 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 325757 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 325757 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 325760 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 325764 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 325765 INFO  (zkCallback-338-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@cb8bb44 name:ZooKeeperConnection Watcher:127.0.0.1:54141/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 325765 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 325765 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 325765 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 325767 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 325768 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 325770 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 325771 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 325771 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 325774 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 325774 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 325776 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 325776 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 325777 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 325778 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 325779 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 325779 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 325781 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 325781 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 325783 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 325783 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 325784 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 325785 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 325786 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 325786 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 325788 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 325788 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 325790 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 325790 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 325792 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 325792 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 325797 INFO  (zkCallback-339-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3162afa4 name:ZooKeeperConnection Watcher:127.0.0.1:54141 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 325797 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 325798 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 325798 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 325800 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 325801 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 325803 INFO  (zkCallback-340-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@12c34ec9 name:ZooKeeperConnection Watcher:127.0.0.1:54141/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 325803 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 325803 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 325804 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf2/solrconfig.xml
   [junit4]   2> 325804 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/solrconfig.xml
   [junit4]   2> 325806 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf2/schema.xml
   [junit4]   2> 325806 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/schema.xml
   [junit4]   2> 325807 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf2/enumsConfig.xml
   [junit4]   2> 325808 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/enumsConfig.xml
   [junit4]   2> 325810 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf2/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 325810 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 325811 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf2/stopwords.txt
   [junit4]   2> 325812 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/stopwords.txt
   [junit4]   2> 325821 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf2/protwords.txt
   [junit4]   2> 325822 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/protwords.txt
   [junit4]   2> 325823 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf2/currency.xml
   [junit4]   2> 325823 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/currency.xml
   [junit4]   2> 325825 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf2/open-exchange-rates.json
   [junit4]   2> 325825 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/open-exchange-rates.json
   [junit4]   2> 325826 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf2/mapping-ISOLatin1Accent.txt
   [junit4]   2> 325826 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/mapping-ISOLatin1Accent.txt
   [junit4]   2> 325828 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf2/old_synonyms.txt
   [junit4]   2> 325828 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/old_synonyms.txt
   [junit4]   2> 325829 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf2/synonyms.txt
   [junit4]   2> 325830 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/synonyms.txt
   [junit4]   2> 325831 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/elevate.xml to /configs/conf2/elevate.xml
   [junit4]   2> 325831 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/elevate.xml
   [junit4]   2> 326142 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_EC5FBD1647494C2F-001/control-001/cores/collection1
   [junit4]   2> 326143 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.e.j.s.Server jetty-9.2.11.v20150529
   [junit4]   2> 326145 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@4a32de0c{/,null,AVAILABLE}
   [junit4]   2> 326146 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.e.j.s.ServerConnector Started ServerConnector@61e4bbe1{HTTP/1.1}{127.0.0.1:58751}
   [junit4]   2> 326146 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.e.j.s.Server Started @329696ms
   [junit4]   2> 326146 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=hdfs://localhost:41535/hdfs__localhost_41535__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-trunk_solr_build_solr-core_test_J1_temp_solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_EC5FBD1647494C2F-001_tempDir-002_control_data, hostContext=/, hostPort=58751, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_EC5FBD1647494C2F-001/control-001/cores}
   [junit4]   2> 326147 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@4e0e2f2a
   [junit4]   2> 326147 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_EC5FBD1647494C2F-001/control-001/'
   [junit4]   2> 326162 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_EC5FBD1647494C2F-001/control-001/solr.xml
   [junit4]   2> 326167 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_EC5FBD1647494C2F-001/control-001/cores
   [junit4]   2> 326167 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.CoreContainer New CoreContainer 1935429587
   [junit4]   2> 326167 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_EC5FBD1647494C2F-001/control-001/]
   [junit4]   2> 326167 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_EC5FBD1647494C2F-001/control-001/lib
   [junit4]   2> 326168 WARN  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_EC5FBD1647494C2F-001/control-001/lib).
   [junit4]   2> 326174 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 326176 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 326176 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 326176 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 326177 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:54141/solr
   [junit4]   2> 326177 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 326177 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 326177 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 326179 INFO  (zkCallback-342-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@75b2e7a9 name:ZooKeeperConnection Watcher:127.0.0.1:54141 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 326179 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 326180 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 326181 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [n:127.0.0.1:58751_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 326182 INFO  (zkCallback-343-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@34dc3f2c name:ZooKeeperConnection Watcher:127.0.0.1:54141/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 326182 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [n:127.0.0.1:58751_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 326184 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [n:127.0.0.1:58751_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 326186 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [n:127.0.0.1:58751_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 326188 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [n:127.0.0.1:58751_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 326190 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [n:127.0.0.1:58751_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 326192 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [n:127.0.0.1:58751_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 326195 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [n:127.0.0.1:58751_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 326197 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [n:127.0.0.1:58751_    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 326199 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [n:127.0.0.1:58751_    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 326200 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [n:127.0.0.1:58751_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:58751_
   [junit4]   2> 326200 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [n:127.0.0.1:58751_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:58751_
   [junit4]   2> 326202 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [n:127.0.0.1:58751_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 326203 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [n:127.0.0.1:58751_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 326208 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [n:127.0.0.1:58751_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 326209 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [n:127.0.0.1:58751_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94228049315233797-127.0.0.1:58751_-n_0000000000
   [junit4]   2> 326210 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [n:127.0.0.1:58751_    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:58751_
   [junit4]   2> 326210 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [n:127.0.0.1:58751_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 326212 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [n:127.0.0.1:58751_    ] o.a.s.c.Overseer Overseer (id=94228049315233797-127.0.0.1:58751_-n_0000000000) starting
   [junit4]   2> 326214 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [n:127.0.0.1:58751_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 326220 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [n:127.0.0.1:58751_    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 326221 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [n:127.0.0.1:58751_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 326221 INFO  (OverseerCollectionProcessor-94228049315233797-127.0.0.1:58751_-n_0000000000) [n:127.0.0.1:58751_    ] o.a.s.c.OverseerCollectionProcessor Process current queue of collection creations
   [junit4]   2> 326221 INFO  (OverseerStateUpdate-94228049315233797-127.0.0.1:58751_-n_0000000000) [n:127.0.0.1:58751_    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 326224 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [n:127.0.0.1:58751_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 326225 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [n:127.0.0.1:58751_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 326225 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [n:127.0.0.1:58751_    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_EC5FBD1647494C2F-001/control-001/cores
   [junit4]   2> 326226 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [n:127.0.0.1:58751_    ] o.a.s.c.SolrCore Created CoreDescriptor: {name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, loadOnStartup=true, configSetProperties=configsetprops.json, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_EC5FBD1647494C2F-001/control-001/cores/collection1, collection=control_collection, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_EC5FBD1647494C2F-001/control-001/cores/collection1/, coreNodeName=, dataDir=data/, shard=}
   [junit4]   2> 326226 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [n:127.0.0.1:58751_    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_EC5FBD1647494C2F-001/control-001/cores/collection1/
   [junit4]   2> 326227 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [n:127.0.0.1:58751_    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 326227 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 326227 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 326228 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [n:127.0.0.1:58751_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1
   [junit4]   2> 326228 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [n:127.0.0.1:58751_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 326228 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 326228 INFO  (zkCallback-343-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_    ] o.a.s.c.DistributedQueue NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 326229 INFO  (OverseerStateUpdate-94228049315233797-127.0.0.1:58751_-n_0000000000) [n:127.0.0.1:58751_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:58751",
   [junit4]   2>   "node_name":"127.0.0.1:58751_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 326229 INFO  (OverseerStateUpdate-94228049315233797-127.0.0.1:58751_-n_0000000000) [n:127.0.0.1:58751_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:58751",
   [junit4]   2>   "node_name":"127.0.0.1:58751_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 326230 INFO  (OverseerStateUpdate-94228049315233797-127.0.0.1:58751_-n_0000000000) [n:127.0.0.1:58751_    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 326230 INFO  (OverseerStateUpdate-94228049315233797-127.0.0.1:58751_-n_0000000000) [n:127.0.0.1:58751_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 326237 INFO  (zkCallback-343-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 326239 INFO  (zkCallback-343-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_    ] o.a.s.c.c.ZkStateReader Updated cluster state version to 1
   [junit4]   2> 327228 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 327229 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 327229 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 327229 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 327230 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 327230 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_EC5FBD1647494C2F-001/control-001/cores/collection1/'
   [junit4]   2> 327254 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 327260 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 327263 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.0.0
   [junit4]   2> 327271 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 327275 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 327281 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 327372 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 327378 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 327379 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 327390 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 327393 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 327395 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 327395 WARN  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 327395 WARN  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 327396 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 327396 WARN  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 327396 WARN  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 327397 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1
   [junit4]   2> 327398 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 327398 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.HdfsDirectoryFactory
   [junit4]   2> 327408 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:41535/solr_hdfs_home
   [junit4]   2> 327409 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
   [junit4]   2> 327409 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_EC5FBD1647494C2F-001/control-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 327409 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@4e98902a
   [junit4]   2> 327410 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:41535/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 327435 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.h.HdfsLocalityReporter Registering direcotry hdfs://localhost:41535/solr_hdfs_home/control_collection/core_node1/data for locality metrics.
   [junit4]   2> 327435 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:41535/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 327440 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=hdfs://localhost:41535/solr_hdfs_home/control_collection/core_node1/data/index/
   [junit4]   2> 327454 WARN  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory 'hdfs:/localhost:41535/solr_hdfs_home/control_collection/core_node1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 327454 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:41535/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 327468 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.h.HdfsLocalityReporter Registering direcotry hdfs://localhost:41535/solr_hdfs_home/control_collection/core_node1/data/index for locality metrics.
   [junit4]   2> 327468 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:41535/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 327469 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=6, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
   [junit4]   2> 327754 INFO  (IPC Server handler 8 on 41535) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:42510 is added to blk_1073741825_1001{blockUCState=COMMITTED, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-09b68d51-a376-4be0-9ee6-b491bd109ec5:NORMAL:127.0.0.1:42510|RBW], ReplicaUnderConstruction[[DISK]DS-3bb541e2-abde-4324-9705-e51f67e84540:NORMAL:127.0.0.1:44118|RBW]]} size 71
   [junit4]   2> 327757 INFO  (IPC Server handler 9 on 41535) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:44118 is added to blk_1073741825_1001 size 71
   [junit4]   2> 328154 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(HdfsDirectory@b6fc8a19 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@734f3b2c; maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 328154 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 328157 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in hdfs://localhost:41535/solr_hdfs_home/control_collection/core_node1/data/
   [junit4]   2> 328161 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 328161 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 328162 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 328162 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 328162 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 328162 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 328162 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 328162 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 328163 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 328165 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 328166 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 328167 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 328168 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 328171 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/mbeans,standard,/update/csv,/update/json/docs,/admin/luke,/admin/segments,/get,/admin/system,/replication,/admin/properties,/config,/schema,/admin/plugins,/admin/logging,/update/json,/admin/threads,/admin/ping,/update,/admin/file
   [junit4]   2> 328172 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 328179 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
   [junit4]   2> 328179 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing HdfsUpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 tlogDfsReplication=2
   [junit4]   2> 328195 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 328195 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 328198 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=39, maxMergeAtOnceExplicit=18, maxMergedSegmentMB=74.4296875, floorSegmentMB=0.7265625, forceMergeDeletesPctAllowed=21.66815875061856, segmentsPerTier=10.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
   [junit4]   2> 328289 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(HdfsDirectory@b6fc8a19 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@734f3b2c; maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 328289 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 328290 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@c263547[collection1] main
   [junit4]   2> 328290 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 328291 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 328291 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 328291 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 328292 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 328292 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 328292 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 328293 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 328293 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 328293 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 328295 INFO  (searcherExecutor-1118-thread-1-processing-n:127.0.0.1:58751_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@c263547[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 328304 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 328304 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 256 version buckets from index
   [junit4]   2> 328304 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 328304 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1507648791828758528
   [junit4]   2> 328304 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 0 ms to seed version buckets with highest version 1507648791828758528
   [junit4]   2> 328304 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 328305 INFO  (coreLoadExecutor-1117-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 328305 INFO  (coreZkRegister-1111-thread-1-processing-n:127.0.0.1:58751_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:58751 collection:control_collection shard:shard1
   [junit4]   2> 328306 INFO  (coreZkRegister-1111-thread-1-processing-n:127.0.0.1:58751_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 328312 INFO  (coreZkRegister-1111-thread-1-processing-n:127.0.0.1:58751_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94228049315233797-core_node1-n_0000000000
   [junit4]   2> 328313 INFO  (coreZkRegister-1111-thread-1-processing-n:127.0.0.1:58751_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 328314 INFO  (zkCallback-343-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_    ] o.a.s.c.DistributedQueue NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 328314 INFO  (coreZkRegister-1111-thread-1-processing-n:127.0.0.1:58751_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 328314 INFO  (coreZkRegister-1111-thread-1-processing-n:127.0.0.1:58751_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 328315 INFO  (OverseerStateUpdate-94228049315233797-127.0.0.1:58751_-n_0000000000) [n:127.0.0.1:58751_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 328315 INFO  (coreZkRegister-1111-thread-1-processing-n:127.0.0.1:58751_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:58751/collection1/
   [junit4]   2> 328315 INFO  (coreZkRegister-1111-thread-1-processing-n:127.0.0.1:58751_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 328315 INFO  (coreZkRegister-1111-thread-1-processing-n:127.0.0.1:58751_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:58751/collection1/ has no replicas
   [junit4]   2> 328315 INFO  (coreZkRegister-1111-thread-1-processing-n:127.0.0.1:58751_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:58751/collection1/ shard1
   [junit4]   2> 328315 INFO  (coreZkRegister-1111-thread-1-processing-n:127.0.0.1:58751_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 328316 INFO  (zkCallback-343-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 328317 INFO  (zkCallback-343-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_    ] o.a.s.c.c.ZkStateReader Updated cluster state version to 2
   [junit4]   2> 328319 INFO  (zkCallback-343-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_    ] o.a.s.c.DistributedQueue NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 328319 INFO  (OverseerStateUpdate-94228049315233797-127.0.0.1:58751_-n_0000000000) [n:127.0.0.1:58751_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:58751",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 2
   [junit4]   2> 328422 INFO  (zkCallback-343-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 328423 INFO  (zkCallback-343-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_    ] o.a.s.c.c.ZkStateReader Updated cluster state version to 3
   [junit4]   2> 328470 INFO  (coreZkRegister-1111-thread-1-processing-n:127.0.0.1:58751_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:58751/collection1/ and leader is http://127.0.0.1:58751/collection1/
   [junit4]   2> 328470 INFO  (coreZkRegister-1111-thread-1-processing-n:127.0.0.1:58751_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:58751
   [junit4]   2> 328470 INFO  (coreZkRegister-1111-thread-1-processing-n:127.0.0.1:58751_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 328470 INFO  (coreZkRegister-1111-thread-1-processing-n:127.0.0.1:58751_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 328471 INFO  (coreZkRegister-1111-thread-1-processing-n:127.0.0.1:58751_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:58751_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 328473 INFO  (zkCallback-343-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_    ] o.a.s.c.DistributedQueue NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 328473 INFO  (OverseerStateUpdate-94228049315233797-127.0.0.1:58751_-n_0000000000) [n:127.0.0.1:58751_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:58751",
   [junit4]   2>   "node_name":"127.0.0.1:58751_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"} current state version: 3
   [junit4]   2> 328474 INFO  (OverseerStateUpdate-94228049315233797-127.0.0.1:58751_-n_0000000000) [n:127.0.0.1:58751_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:58751",
   [junit4]   2>   "node_name":"127.0.0.1:58751_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 328576 INFO  (zkCallback-343-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 328578 INFO  (zkCallback-343-thread-1-processing-n:127.0.0.1:58751_) [n:127.0.0.1:58751_    ] o.a.s.c.c.ZkStateReader Updated cluster state version to 4
   [junit4]   2> 328730 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 328731 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 328732 INFO  (zkCallback-345-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5796dc1c name:ZooKeeperConnection Watcher:127.0.0.1:54141/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 328733 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 328733 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 328733 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 328737 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 329054 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_EC5FBD1647494C2F-001/shard-1-001/cores/collection1
   [junit4]   2> 329055 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_EC5FBD1647494C2F-001/shard-1-001
   [junit4]   2> 329056 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.e.j.s.Server jetty-9.2.11.v20150529
   [junit4]   2> 329058 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@34c1bbe{/,null,AVAILABLE}
   [junit4]   2> 329058 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.e.j.s.ServerConnector Started ServerConnector@24af1712{HTTP/1.1}{127.0.0.1:38533}
   [junit4]   2> 329058 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.e.j.s.Server Started @332608ms
   [junit4]   2> 329058 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=hdfs://localhost:41535/hdfs__localhost_41535__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-trunk_solr_build_solr-core_test_J1_temp_solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_EC5FBD1647494C2F-001_tempDir-002_jetty1, solrconfig=solrconfig.xml, hostContext=/, hostPort=38533, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_EC5FBD1647494C2F-001/shard-1-001/cores}
   [junit4]   2> 329059 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@4e0e2f2a
   [junit4]   2> 329059 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_EC5FBD1647494C2F-001/shard-1-001/'
   [junit4]   2> 329074 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_EC5FBD1647494C2F-001/shard-1-001/solr.xml
   [junit4]   2> 329079 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_EC5FBD1647494C2F-001/shard-1-001/cores
   [junit4]   2> 329079 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.CoreContainer New CoreContainer 1236840958
   [junit4]   2> 329079 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_EC5FBD1647494C2F-001/shard-1-001/]
   [junit4]   2> 329079 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_EC5FBD1647494C2F-001/shard-1-001/lib
   [junit4]   2> 329079 WARN  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_EC5FBD1647494C2F-001/shard-1-001/lib).
   [junit4]   2> 329086 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 329088 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 329088 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 329088 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 329088 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:54141/solr
   [junit4]   2> 329089 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 329089 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 329089 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 329091 INFO  (zkCallback-347-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@25a149e6 name:ZooKeeperConnection Watcher:127.0.0.1:54141 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 329091 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 329091 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 329095 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [n:127.0.0.1:38533_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 329096 INFO  (zkCallback-348-thread-1-processing-n:127.0.0.1:38533_) [n:127.0.0.1:38533_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5f7381c3 name:ZooKeeperConnection Watcher:127.0.0.1:54141/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 329096 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [n:127.0.0.1:38533_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 329100 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [n:127.0.0.1:38533_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 330104 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [n:127.0.0.1:38533_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:38533_
   [junit4]   2> 330105 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [n:127.0.0.1:38533_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:38533_
   [junit4]   2> 330108 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [n:127.0.0.1:38533_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 330109 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [n:127.0.0.1:38533_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94228049315233800-127.0.0.1:38533_-n_0000000001
   [junit4]   2> 330110 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [n:127.0.0.1:38533_    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/94228049315233797-127.0.0.1:58751_-n_0000000000 to know if I could be the leader
   [junit4]   2> 330110 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [n:127.0.0.1:38533_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 330111 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [n:127.0.0.1:38533_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 330111 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [n:127.0.0.1:38533_    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_EC5FBD1647494C2F-001/shard-1-001/cores
   [junit4]   2> 330112 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) [n:127.0.0.1:38533_    ] o.a.s.c.SolrCore Created CoreDescriptor: {name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, loadOnStartup=true, configSetProperties=configsetprops.json, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_EC5FBD1647494C2F-001/shard-1-001/cores/collection1, collection=collection1, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_EC5FBD1647494C2F-001/shard-1-001/cores/collection1/, coreNodeName=, dataDir=data/, shard=}
   [junit4]   2> 330112 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[EC5FBD1647494C2F]) 

[...truncated too long message...]

.handler.TestReplicationHandler_EC5FBD1647494C2F-001/solr-instance-026/./collection1/data;done=false>>]
   [junit4]   2> 2840731 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[EC5FBD1647494C2F]) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_EC5FBD1647494C2F-001/solr-instance-026/./collection1/data
   [junit4]   2> 2840732 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[EC5FBD1647494C2F]) [    x:collection1] o.a.s.c.CachingDirectoryFactory looking to close /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_EC5FBD1647494C2F-001/solr-instance-026/./collection1/data/index [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_EC5FBD1647494C2F-001/solr-instance-026/./collection1/data/index;done=false>>]
   [junit4]   2> 2840732 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[EC5FBD1647494C2F]) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_EC5FBD1647494C2F-001/solr-instance-026/./collection1/data/index
   [junit4]   2> 2840732 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[EC5FBD1647494C2F]) [    x:collection1] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@779e9c4c{/solr,null,UNAVAILABLE}
   [junit4]   2> 2840746 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[EC5FBD1647494C2F]) [    ] o.e.j.s.ServerConnector Stopped ServerConnector@5c0dfc2e{HTTP/1.1}{127.0.0.1:0}
   [junit4]   2> 2840747 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[EC5FBD1647494C2F]) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1132517580
   [junit4]   2> 2840751 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[EC5FBD1647494C2F]) [    x:collection1] o.a.s.c.SolrCore [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@2e6851c6
   [junit4]   2> 2840751 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[EC5FBD1647494C2F]) [    x:collection1] o.a.s.u.UpdateHandler closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
   [junit4]   2> 2840751 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[EC5FBD1647494C2F]) [    x:collection1] o.a.s.u.SolrCoreState Closing SolrCoreState
   [junit4]   2> 2840751 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[EC5FBD1647494C2F]) [    x:collection1] o.a.s.u.DefaultSolrCoreState SolrCoreState ref count has reached 0 - closing IndexWriter
   [junit4]   2> 2840751 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[EC5FBD1647494C2F]) [    x:collection1] o.a.s.u.DefaultSolrCoreState closing IndexWriter with IndexWriterCloser
   [junit4]   2> 2840755 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[EC5FBD1647494C2F]) [    x:collection1] o.a.s.c.SolrCore [collection1] Closing main searcher on request.
   [junit4]   2> 2840755 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[EC5FBD1647494C2F]) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing MockDirectoryFactory - 2 directories currently being tracked
   [junit4]   2> 2840755 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[EC5FBD1647494C2F]) [    x:collection1] o.a.s.c.CachingDirectoryFactory looking to close /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_EC5FBD1647494C2F-001/solr-instance-027/./collection1/data [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_EC5FBD1647494C2F-001/solr-instance-027/./collection1/data;done=false>>]
   [junit4]   2> 2840755 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[EC5FBD1647494C2F]) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_EC5FBD1647494C2F-001/solr-instance-027/./collection1/data
   [junit4]   2> 2840756 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[EC5FBD1647494C2F]) [    x:collection1] o.a.s.c.CachingDirectoryFactory looking to close /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_EC5FBD1647494C2F-001/solr-instance-027/./collection1/data/index [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_EC5FBD1647494C2F-001/solr-instance-027/./collection1/data/index;done=false>>]
   [junit4]   2> 2840756 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[EC5FBD1647494C2F]) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_EC5FBD1647494C2F-001/solr-instance-027/./collection1/data/index
   [junit4]   2> 2840757 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[EC5FBD1647494C2F]) [    x:collection1] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@11890b97{/solr,null,UNAVAILABLE}
   [junit4]   2> 2840758 INFO  (SUITE-TestReplicationHandler-seed#[EC5FBD1647494C2F]-worker) [    ] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_EC5FBD1647494C2F-001
   [junit4]   2> NOTE: test params are: codec=CheapBastard, sim=DefaultSimilarity, locale=ar_IQ, timezone=Asia/Dacca
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.8.0_45 (64-bit)/cpus=4,threads=1,free=324170384,total=510656512
   [junit4]   2> NOTE: All tests run in this JVM: [ZkNodePropsTest, ConnectionManagerTest, RollingRestartTest, SignatureUpdateProcessorFactoryTest, CustomCollectionTest, DistributedFacetPivotLongTailTest, TestCodecSupport, TestBinaryField, SuggesterFSTTest, ResponseHeaderTest, TestDocBasedVersionConstraints, TestIBSimilarityFactory, TestJettySolrRunner, AssignTest, SolrCoreTest, UniqFieldsUpdateProcessorFactoryTest, TestSchemaNameResource, SmileWriterTest, TestSolrDynamicMBean, BlockDirectoryTest, TestConfig, IndexBasedSpellCheckerTest, TestJmxIntegration, TestRebalanceLeaders, TestUpdate, TestMiniSolrCloudCluster, TestDynamicFieldResource, TestDynamicLoading, ChangedSchemaMergeTest, SharedFSAutoReplicaFailoverUtilsTest, PreAnalyzedUpdateProcessorTest, DataDrivenBlockJoinTest, DocExpirationUpdateProcessorFactoryTest, URLClassifyProcessorTest, TestArbitraryIndexDir, ChaosMonkeyNothingIsSafeTest, TestClusterStateMutator, SimpleCollectionCreateDeleteTest, RemoteQueryErrorTest, TestConfigReload, TestCloudPivotFacet, UUIDFieldTest, DistributedSpellCheckComponentTest, SegmentsInfoRequestHandlerTest, TestSimpleTrackingShardHandler, TestSchemaManager, TestSQLHandler, TestCollapseQParserPlugin, BasicDistributedZkTest, DistributedFacetPivotWhiteBoxTest, PeerSyncTest, SolrRequestParserTest, SolrIndexConfigTest, TestPhraseSuggestions, MinimalSchemaTest, TestChildDocTransformer, TestRequestStatusCollectionAPI, CdcrReplicationHandlerTest, TestRTimer, SuggesterWFSTTest, TermVectorComponentTest, TestManagedSynonymFilterFactory, TestPivotHelperCode, BJQParserTest, TestFaceting, SchemaVersionSpecificBehaviorTest, TestFreeTextSuggestions, TestMergePolicyConfig, TestFileDictionaryLookup, TestClassNameShortening, CollectionTooManyReplicasTest, SyncSliceTest, RecoveryAfterSoftCommitTest, TestStressLucene, TestSolrDeletionPolicy1, TestWordDelimiterFilterFactory, TestJsonFacets, TestRandomFaceting, FastVectorHighlighterTest, QueryElevationComponentTest, VersionInfoTest, TestManagedResource, HLLSerializationTest, TermsComponentTest, CursorMarkTest, BasicDistributedZk2Test, LeaderElectionIntegrationTest, BasicZkTest, TestReplicationHandler]
   [junit4] Completed [276/525] on J0 in 547.05s, 13 tests, 1 failure <<< FAILURES!

[...truncated 778 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:721: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:658: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build.xml:230: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/common-build.xml:517: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/common-build.xml:1429: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/common-build.xml:984: There were test failures: 525 suites, 2071 tests, 1 error, 1 failure, 20 ignored (10 assumptions)

Total time: 171 minutes 27 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
No prior successful build to compare, so performing full copy of artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-NightlyTests-trunk - Build # 745 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-trunk/745/

No tests ran.

Build Log:
[...truncated 1019 lines...]
FATAL: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel
hudson.remoting.RequestAbortedException: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel
	at hudson.remoting.RequestAbortedException.wrapForRethrow(RequestAbortedException.java:41)
	at hudson.remoting.RequestAbortedException.wrapForRethrow(RequestAbortedException.java:34)
	at hudson.remoting.Request.call(Request.java:174)
	at hudson.remoting.Channel.call(Channel.java:742)
	at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:168)
	at com.sun.proxy.$Proxy59.join(Unknown Source)
	at hudson.Launcher$RemoteLauncher$ProcImpl.join(Launcher.java:956)
	at hudson.Launcher$ProcStarter.join(Launcher.java:367)
	at hudson.tasks.Ant.perform(Ant.java:217)
	at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
	at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:756)
	at hudson.model.Build$BuildExecution.build(Build.java:198)
	at hudson.model.Build$BuildExecution.doRun(Build.java:159)
	at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:529)
	at hudson.model.Run.execute(Run.java:1706)
	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
	at hudson.model.ResourceController.execute(ResourceController.java:88)
	at hudson.model.Executor.run(Executor.java:232)
Caused by: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel
	at hudson.remoting.Request.abort(Request.java:299)
	at hudson.remoting.Channel.terminate(Channel.java:805)
	at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:69)
Caused by: java.io.IOException: Unexpected termination of the channel
	at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:50)
Caused by: java.io.EOFException
	at java.io.ObjectInputStream$PeekInputStream.readFully(ObjectInputStream.java:2325)
	at java.io.ObjectInputStream$BlockDataInputStream.readShort(ObjectInputStream.java:2794)
	at java.io.ObjectInputStream.readStreamHeader(ObjectInputStream.java:801)
	at java.io.ObjectInputStream.<init>(ObjectInputStream.java:299)
	at hudson.remoting.ObjectInputStreamEx.<init>(ObjectInputStreamEx.java:40)
	at hudson.remoting.AbstractSynchronousByteArrayCommandTransport.read(AbstractSynchronousByteArrayCommandTransport.java:34)
	at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:48)



[JENKINS] Lucene-Solr-NightlyTests-trunk - Build # 744 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-trunk/744/

2 tests failed.
FAILED:  org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=85454, name=collection0, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=85454, name=collection0, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:33836/pwz/ls: Could not find collection : awholynewstresscollection_collection0_0
	at __randomizedtesting.SeedInfo.seed([B739B805AAD08D6E]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:560)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:234)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:226)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:376)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:328)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1086)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:856)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:799)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:894)


FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
KeeperErrorCode = Session expired for /live_nodes

Stack Trace:
org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /live_nodes
	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
	at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1472)
	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:336)
	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:333)
	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:61)
	at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:333)
	at org.apache.solr.common.cloud.ZkStateReader.updateClusterState(ZkStateReader.java:496)
	at org.apache.solr.common.cloud.ZkStateReader.updateClusterState(ZkStateReader.java:229)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testNoCollectionSpecified(CollectionsAPIDistributedZkTest.java:464)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test(CollectionsAPIDistributedZkTest.java:169)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:497)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:960)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:935)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 10796 lines...]
   [junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_B739B805AAD08D6E-001/init-core-data-001
   [junit4]   2> 1853801 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[B739B805AAD08D6E]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false)
   [junit4]   2> 1853801 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[B739B805AAD08D6E]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /yv_/ng
   [junit4]   2> 1853804 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 1853804 INFO  (Thread-15001) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1853805 INFO  (Thread-15001) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 1853905 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.c.ZkTestServer start zk server on port:37397
   [junit4]   2> 1853905 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1853907 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1853910 INFO  (zkCallback-2133-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@33e6a5d1 name:ZooKeeperConnection Watcher:127.0.0.1:37397 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1853910 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1853911 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1853911 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 1853914 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1853915 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1853916 INFO  (zkCallback-2134-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2a3fa9ee name:ZooKeeperConnection Watcher:127.0.0.1:37397/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1853916 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1853916 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1853916 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 1853919 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 1853921 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 1853923 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 1853925 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 1853926 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 1853929 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 1853930 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 1853932 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1853932 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1853934 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 1853935 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 1853937 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 1853937 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 1853940 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 1853940 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 1853942 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 1853942 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 1853945 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1853945 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1853947 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1853948 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1853950 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 1853951 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 1853953 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 1853953 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 1853957 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1853957 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1853958 INFO  (zkCallback-2135-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@a183aff name:ZooKeeperConnection Watcher:127.0.0.1:37397 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1853959 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1853959 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1853959 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 1853961 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1853962 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1853963 INFO  (zkCallback-2136-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2db3668d name:ZooKeeperConnection Watcher:127.0.0.1:37397/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1853963 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1853964 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1853965 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf2/solrconfig.xml
   [junit4]   2> 1853965 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/solrconfig.xml
   [junit4]   2> 1853968 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf2/schema.xml
   [junit4]   2> 1853968 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/schema.xml
   [junit4]   2> 1853971 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf2/enumsConfig.xml
   [junit4]   2> 1853971 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/enumsConfig.xml
   [junit4]   2> 1853974 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf2/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1853974 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1853977 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf2/stopwords.txt
   [junit4]   2> 1853977 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/stopwords.txt
   [junit4]   2> 1853980 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf2/protwords.txt
   [junit4]   2> 1853980 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/protwords.txt
   [junit4]   2> 1853982 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf2/currency.xml
   [junit4]   2> 1853982 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/currency.xml
   [junit4]   2> 1853984 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf2/open-exchange-rates.json
   [junit4]   2> 1853985 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/open-exchange-rates.json
   [junit4]   2> 1853987 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf2/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1853987 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1853989 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf2/old_synonyms.txt
   [junit4]   2> 1853989 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/old_synonyms.txt
   [junit4]   2> 1853991 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf2/synonyms.txt
   [junit4]   2> 1853991 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/synonyms.txt
   [junit4]   2> 1853993 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/elevate.xml to /configs/conf2/elevate.xml
   [junit4]   2> 1853994 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/elevate.xml
   [junit4]   2> 1854341 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_B739B805AAD08D6E-001/control-001/cores/collection1
   [junit4]   2> 1854343 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.e.j.s.Server jetty-9.2.11.v20150529
   [junit4]   2> 1854345 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@49c90a92{/yv_/ng,null,AVAILABLE}
   [junit4]   2> 1854345 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.e.j.s.ServerConnector Started ServerConnector@4ba7ee6a{HTTP/1.1}{127.0.0.1:40156}
   [junit4]   2> 1854346 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.e.j.s.Server Started @1858158ms
   [junit4]   2> 1854346 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_B739B805AAD08D6E-001/tempDir-001/control/data, hostContext=/yv_/ng, hostPort=40156, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_B739B805AAD08D6E-001/control-001/cores}
   [junit4]   2> 1854346 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@4e0e2f2a
   [junit4]   2> 1854346 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_B739B805AAD08D6E-001/control-001/'
   [junit4]   2> 1854364 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_B739B805AAD08D6E-001/control-001/solr.xml
   [junit4]   2> 1854370 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_B739B805AAD08D6E-001/control-001/cores
   [junit4]   2> 1854370 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.c.CoreContainer New CoreContainer 181475766
   [junit4]   2> 1854370 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_B739B805AAD08D6E-001/control-001/]
   [junit4]   2> 1854370 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_B739B805AAD08D6E-001/control-001/lib
   [junit4]   2> 1854371 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_B739B805AAD08D6E-001/control-001/lib).
   [junit4]   2> 1854379 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1854380 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1854381 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1854381 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1854381 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:37397/solr
   [junit4]   2> 1854381 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1854381 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1854382 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1854384 INFO  (zkCallback-2138-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@67fed3cb name:ZooKeeperConnection Watcher:127.0.0.1:37397 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1854384 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1854384 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1854387 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [n:127.0.0.1:40156_yv_%2Fng    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1854388 INFO  (zkCallback-2139-thread-1-processing-n:127.0.0.1:40156_yv_%2Fng) [n:127.0.0.1:40156_yv_%2Fng    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@34966093 name:ZooKeeperConnection Watcher:127.0.0.1:37397/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1854388 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [n:127.0.0.1:40156_yv_%2Fng    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1854390 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [n:127.0.0.1:40156_yv_%2Fng    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 1854393 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [n:127.0.0.1:40156_yv_%2Fng    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 1854395 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [n:127.0.0.1:40156_yv_%2Fng    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 1854398 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [n:127.0.0.1:40156_yv_%2Fng    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 1854400 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [n:127.0.0.1:40156_yv_%2Fng    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 1854403 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [n:127.0.0.1:40156_yv_%2Fng    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 1854405 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [n:127.0.0.1:40156_yv_%2Fng    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 1854407 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [n:127.0.0.1:40156_yv_%2Fng    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 1854408 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [n:127.0.0.1:40156_yv_%2Fng    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:40156_yv_%2Fng
   [junit4]   2> 1854409 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [n:127.0.0.1:40156_yv_%2Fng    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:40156_yv_%2Fng
   [junit4]   2> 1854410 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [n:127.0.0.1:40156_yv_%2Fng    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 1854412 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [n:127.0.0.1:40156_yv_%2Fng    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 1854413 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [n:127.0.0.1:40156_yv_%2Fng    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1854414 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [n:127.0.0.1:40156_yv_%2Fng    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94194263117791237-127.0.0.1:40156_yv_%2Fng-n_0000000000
   [junit4]   2> 1854415 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [n:127.0.0.1:40156_yv_%2Fng    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:40156_yv_%2Fng
   [junit4]   2> 1854415 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [n:127.0.0.1:40156_yv_%2Fng    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 1854417 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [n:127.0.0.1:40156_yv_%2Fng    ] o.a.s.c.Overseer Overseer (id=94194263117791237-127.0.0.1:40156_yv_%2Fng-n_0000000000) starting
   [junit4]   2> 1854419 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [n:127.0.0.1:40156_yv_%2Fng    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 1854425 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [n:127.0.0.1:40156_yv_%2Fng    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 1854426 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [n:127.0.0.1:40156_yv_%2Fng    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1854426 INFO  (OverseerCollectionProcessor-94194263117791237-127.0.0.1:40156_yv_%2Fng-n_0000000000) [n:127.0.0.1:40156_yv_%2Fng    ] o.a.s.c.OverseerCollectionProcessor Process current queue of collection creations
   [junit4]   2> 1854426 INFO  (OverseerStateUpdate-94194263117791237-127.0.0.1:40156_yv_%2Fng-n_0000000000) [n:127.0.0.1:40156_yv_%2Fng    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 1854430 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [n:127.0.0.1:40156_yv_%2Fng    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1854430 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [n:127.0.0.1:40156_yv_%2Fng    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1854431 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [n:127.0.0.1:40156_yv_%2Fng    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_B739B805AAD08D6E-001/control-001/cores
   [junit4]   2> 1854432 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [n:127.0.0.1:40156_yv_%2Fng    ] o.a.s.c.SolrCore Created CoreDescriptor: {name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, loadOnStartup=true, configSetProperties=configsetprops.json, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_B739B805AAD08D6E-001/control-001/cores/collection1, collection=control_collection, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_B739B805AAD08D6E-001/control-001/cores/collection1/, coreNodeName=, dataDir=data/, shard=}
   [junit4]   2> 1854433 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [n:127.0.0.1:40156_yv_%2Fng    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_B739B805AAD08D6E-001/control-001/cores/collection1/
   [junit4]   2> 1854433 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [n:127.0.0.1:40156_yv_%2Fng    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 1854434 INFO  (coreLoadExecutor-5175-thread-1-processing-n:127.0.0.1:40156_yv_%2Fng) [n:127.0.0.1:40156_yv_%2Fng c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 1854434 INFO  (coreLoadExecutor-5175-thread-1-processing-n:127.0.0.1:40156_yv_%2Fng) [n:127.0.0.1:40156_yv_%2Fng c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1854435 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [n:127.0.0.1:40156_yv_%2Fng    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2
   [junit4]   2> 1854435 INFO  (coreLoadExecutor-5175-thread-1-processing-n:127.0.0.1:40156_yv_%2Fng) [n:127.0.0.1:40156_yv_%2Fng c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 1854435 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [n:127.0.0.1:40156_yv_%2Fng    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1854435 INFO  (zkCallback-2139-thread-1-processing-n:127.0.0.1:40156_yv_%2Fng) [n:127.0.0.1:40156_yv_%2Fng    ] o.a.s.c.DistributedQueue NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 1854436 INFO  (OverseerStateUpdate-94194263117791237-127.0.0.1:40156_yv_%2Fng-n_0000000000) [n:127.0.0.1:40156_yv_%2Fng    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:40156/yv_/ng",
   [junit4]   2>   "node_name":"127.0.0.1:40156_yv_%2Fng",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 1854436 INFO  (OverseerStateUpdate-94194263117791237-127.0.0.1:40156_yv_%2Fng-n_0000000000) [n:127.0.0.1:40156_yv_%2Fng    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:40156/yv_/ng",
   [junit4]   2>   "node_name":"127.0.0.1:40156_yv_%2Fng",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 1854437 INFO  (OverseerStateUpdate-94194263117791237-127.0.0.1:40156_yv_%2Fng-n_0000000000) [n:127.0.0.1:40156_yv_%2Fng    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 1854437 INFO  (OverseerStateUpdate-94194263117791237-127.0.0.1:40156_yv_%2Fng-n_0000000000) [n:127.0.0.1:40156_yv_%2Fng    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 1854438 INFO  (zkCallback-2139-thread-1-processing-n:127.0.0.1:40156_yv_%2Fng) [n:127.0.0.1:40156_yv_%2Fng    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1854439 INFO  (zkCallback-2139-thread-1-processing-n:127.0.0.1:40156_yv_%2Fng) [n:127.0.0.1:40156_yv_%2Fng    ] o.a.s.c.c.ZkStateReader Updated cluster state version to 1
   [junit4]   2> 1855435 INFO  (coreLoadExecutor-5175-thread-1-processing-n:127.0.0.1:40156_yv_%2Fng) [n:127.0.0.1:40156_yv_%2Fng c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1855435 INFO  (coreLoadExecutor-5175-thread-1-processing-n:127.0.0.1:40156_yv_%2Fng) [n:127.0.0.1:40156_yv_%2Fng c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 1855436 INFO  (coreLoadExecutor-5175-thread-1-processing-n:127.0.0.1:40156_yv_%2Fng) [n:127.0.0.1:40156_yv_%2Fng c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 1855436 INFO  (coreLoadExecutor-5175-thread-1-processing-n:127.0.0.1:40156_yv_%2Fng) [n:127.0.0.1:40156_yv_%2Fng c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 1855437 INFO  (coreLoadExecutor-5175-thread-1-processing-n:127.0.0.1:40156_yv_%2Fng) [n:127.0.0.1:40156_yv_%2Fng c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1855437 INFO  (coreLoadExecutor-5175-thread-1-processing-n:127.0.0.1:40156_yv_%2Fng) [n:127.0.0.1:40156_yv_%2Fng c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_B739B805AAD08D6E-001/control-001/cores/collection1/'
   [junit4]   2> 1855452 INFO  (coreLoadExecutor-5175-thread-1-processing-n:127.0.0.1:40156_yv_%2Fng) [n:127.0.0.1:40156_yv_%2Fng c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 1855457 INFO  (coreLoadExecutor-5175-thread-1-processing-n:127.0.0.1:40156_yv_%2Fng) [n:127.0.0.1:40156_yv_%2Fng c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 1855461 INFO  (coreLoadExecutor-5175-thread-1-processing-n:127.0.0.1:40156_yv_%2Fng) [n:127.0.0.1:40156_yv_%2Fng c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.0.0
   [junit4]   2> 1855470 INFO  (coreLoadExecutor-5175-thread-1-processing-n:127.0.0.1:40156_yv_%2Fng) [n:127.0.0.1:40156_yv_%2Fng c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1855472 INFO  (coreLoadExecutor-5175-thread-1-processing-n:127.0.0.1:40156_yv_%2Fng) [n:127.0.0.1:40156_yv_%2Fng c:control_collection   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 1855477 INFO  (coreLoadExecutor-5175-thread-1-processing-n:127.0.0.1:40156_yv_%2Fng) [n:127.0.0.1:40156_yv_%2Fng c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1855600 INFO  (coreLoadExecutor-5175-thread-1-processing-n:127.0.0.1:40156_yv_%2Fng) [n:127.0.0.1:40156_yv_%2Fng c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 1855606 INFO  (coreLoadExecutor-5175-thread-1-processing-n:127.0.0.1:40156_yv_%2Fng) [n:127.0.0.1:40156_yv_%2Fng c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 1855607 INFO  (coreLoadExecutor-5175-thread-1-processing-n:127.0.0.1:40156_yv_%2Fng) [n:127.0.0.1:40156_yv_%2Fng c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 1855616 INFO  (coreLoadExecutor-5175-thread-1-processing-n:127.0.0.1:40156_yv_%2Fng) [n:127.0.0.1:40156_yv_%2Fng c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1855630 INFO  (coreLoadExecutor-5175-thread-1-processing-n:127.0.0.1:40156_yv_%2Fng) [n:127.0.0.1:40156_yv_%2Fng c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1855633 INFO  (coreLoadExecutor-5175-thread-1-processing-n:127.0.0.1:40156_yv_%2Fng) [n:127.0.0.1:40156_yv_%2Fng c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1855633 WARN  (coreLoadExecutor-5175-thread-1-processing-n:127.0.0.1:40156_yv_%2Fng) [n:127.0.0.1:40156_yv_%2Fng c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 1855634 WARN  (coreLoadExecutor-5175-thread-1-processing-n:127.0.0.1:40156_yv_%2Fng) [n:127.0.0.1:40156_yv_%2Fng c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 1855634 INFO  (coreLoadExecutor-5175-thread-1-processing-n:127.0.0.1:40156_yv_%2Fng) [n:127.0.0.1:40156_yv_%2Fng c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1855634 WARN  (coreLoadExecutor-5175-thread-1-processing-n:127.0.0.1:40156_yv_%2Fng) [n:127.0.0.1:40156_yv_%2Fng c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 1855635 WARN  (coreLoadExecutor-5175-thread-1-processing-n:127.0.0.1:40156_yv_%2Fng) [n:127.0.0.1:40156_yv_%2Fng c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 1855636 INFO  (coreLoadExecutor-5175-thread-1-processing-n:127.0.0.1:40156_yv_%2Fng) [n:127.0.0.1:40156_yv_%2Fng c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties
   [junit4]   2> org.apache.solr.core.SolrResourceNotFoundException: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2
   [junit4]   2> 	at org.apache.solr.cloud.ZkSolrResourceLoader.openResource(ZkSolrResourceLoader.java:99)
   [junit4]   2> 	at org.apache.solr.core.ConfigSetProperties.readFromResourceLoader(ConfigSetProperties.java:49)
   [junit4]   2> 	at org.apache.solr.core.ConfigSetService.createConfigSetProperties(ConfigSetService.java:114)
   [junit4]   2> 	at org.apache.solr.core.ConfigSetService.getConfig(ConfigSetService.java:76)
   [junit4]   2> 	at org.apache.solr.core.CoreContainer.create(CoreContainer.java:668)
   [junit4]   2> 	at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:397)
   [junit4]   2> 	at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:388)
   [junit4]   2> 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
   [junit4]   2> 	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:156)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 1855636 INFO  (coreLoadExecutor-5175-thread-1-processing-n:127.0.0.1:40156_yv_%2Fng) [n:127.0.0.1:40156_yv_%2Fng c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 1855636 INFO  (coreLoadExecutor-5175-thread-1-processing-n:127.0.0.1:40156_yv_%2Fng) [n:127.0.0.1:40156_yv_%2Fng c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 1855637 INFO  (coreLoadExecutor-5175-thread-1-processing-n:127.0.0.1:40156_yv_%2Fng) [n:127.0.0.1:40156_yv_%2Fng c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_B739B805AAD08D6E-001/control-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 1855637 INFO  (coreLoadExecutor-5175-thread-1-processing-n:127.0.0.1:40156_yv_%2Fng) [n:127.0.0.1:40156_yv_%2Fng c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@522663db
   [junit4]   2> 1855637 INFO  (coreLoadExecutor-5175-thread-1-processing-n:127.0.0.1:40156_yv_%2Fng) [n:127.0.0.1:40156_yv_%2Fng c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_B739B805AAD08D6E-001/control-001/cores/collection1/data
   [junit4]   2> 1855638 INFO  (coreLoadExecutor-5175-thread-1-processing-n:127.0.0.1:40156_yv_%2Fng) [n:127.0.0.1:40156_yv_%2Fng c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_B739B805AAD08D6E-001/control-001/cores/collection1/data/index/
   [junit4]   2> 1855638 WARN  (coreLoadExecutor-5175-thread-1-processing-n:127.0.0.1:40156_yv_%2Fng) [n:127.0.0.1:40156_yv_%2Fng c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_B739B805AAD08D6E-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 1855638 INFO  (coreLoadExecutor-5175-thread-1-processing-n:127.0.0.1:40156_yv_%2Fng) [n:127.0.0.1:40156_yv_%2Fng c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_B739B805AAD08D6E-001/control-001/cores/collection1/data/index
   [junit4]   2> 1855638 INFO  (coreLoadExecutor-5175-thread-1-processing-n:127.0.0.1:40156_yv_%2Fng) [n:127.0.0.1:40156_yv_%2Fng c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=3, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.6198870484959408]
   [junit4]   2> 1855639 INFO  (coreLoadExecutor-5175-thread-1-processing-n:127.0.0.1:40156_yv_%2Fng) [n:127.0.0.1:40156_yv_%2Fng c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(NRTCachingDirectory(RAMDirectory@77689c4e lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@61f51f70; maxCacheMB=0.62109375 maxMergeSizeMB=0.955078125)),segFN=segments_1,generation=1}
   [junit4]   2> 1855639 INFO  (coreLoadExecutor-5175-thread-1-processing-n:127.0.0.1:40156_yv_%2Fng) [n:127.0.0.1:40156_yv_%2Fng c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 1855640 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:40156_yv_%2Fng c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_B739B805AAD08D6E-001/control-001/cores/collection1/data/
   [junit4]   2> 1855640 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:40156_yv_%2Fng c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_B739B805AAD08D6E-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 1855644 INFO  (coreLoadExecutor-5175-thread-1-processing-n:127.0.0.1:40156_yv_%2Fng) [n:127.0.0.1:40156_yv_%2Fng c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 1855644 INFO  (coreLoadExecutor-5175-thread-1-processing-n:127.0.0.1:40156_yv_%2Fng) [n:127.0.0.1:40156_yv_%2Fng c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 1855644 INFO  (coreLoadExecutor-5175-thread-1-processing-n:127.0.0.1:40156_yv_%2Fng) [n:127.0.0.1:40156_yv_%2Fng c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 1855644 INFO  (coreLoadExecutor-5175-thread-1-processing-n:127.0.0.1:40156_yv_%2Fng) [n:127.0.0.1:40156_yv_%2Fng c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1855645 INFO  (coreLoadExecutor-5175-thread-1-processing-n:127.0.0.1:40156_yv_%2Fng) [n:127.0.0.1:40156_yv_%2Fng c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1855645 INFO  (coreLoadExecutor-5175-thread-1-processing-n:127.0.0.1:40156_yv_%2Fng) [n:127.0.0.1:40156_yv_%2Fng c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 1855645 INFO  (coreLoadExecutor-5175-thread-1-processing-n:127.0.0.1:40156_yv_%2Fng) [n:127.0.0.1:40156_yv_%2Fng c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1855645 INFO  (coreLoadExecutor-5175-thread-1-processing-n:127.0.0.1:40156_yv_%2Fng) [n:127.0.0.1:40156_yv_%2Fng c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1855645 INFO  (coreLoadExecutor-5175-thread-1-processing-n:127.0.0.1:40156_yv_%2Fng) [n:127.0.0.1:40156_yv_%2Fng c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 1855647 INFO  (coreLoadExecutor-5175-thread-1-processing-n:127.0.0.1:40156_yv_%2Fng) [n:127.0.0.1:40156_yv_%2Fng c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1855648 INFO  (coreLoadExecutor-5175-thread-1-processing-n:127.0.0.1:40156_yv_%2Fng) [n:127.0.0.1:40156_yv_%2Fng c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1855649 INFO  (coreLoadExecutor-5175-thread-1-processing-n:127.0.0.1:40156_yv_%2Fng) [n:127.0.0.1:40156_yv_%2Fng c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1855650 INFO  (coreLoadExecutor-5175-thread-1-processing-n:127.0.0.1:40156_yv_%2Fng) [n:127.0.0.1:40156_yv_%2Fng c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1855653 INFO  (coreLoadExecutor-5175-thread-1-processing-n:127.0.0.1:40156_yv_%2Fng) [n:127.0.0.1:40156_yv_%2Fng c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/mbeans,standard,/update/csv,/update/json/docs,/admin/luke,/admin/segments,/get,/admin/system,/replication,/admin/properties,/config,/schema,/admin/plugins,/admin/logging,/update/json,/admin/threads,/admin/ping,/update,/admin/file
   [junit4]   2> 1855654 INFO  (coreLoadExecutor-5175-thread-1-processing-n:127.0.0.1:40156_yv_%2Fng) [n:127.0.0.1:40156_yv_%2Fng c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 1855655 INFO  (coreLoadExecutor-5175-thread-1-processing-n:127.0.0.1:40156_yv_%2Fng) [n:127.0.0.1:40156_yv_%2Fng c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1855655 INFO  (coreLoadExecutor-5175-thread-1-processing-n:127.0.0.1:40156_yv_%2Fng) [n:127.0.0.1:40156_yv_%2Fng c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1855656 INFO  (coreLoadExecutor-5175-thread-1-processing-n:127.0.0.1:40156_yv_%2Fng) [n:127.0.0.1:40156_yv_%2Fng c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 1855656 INFO  (coreLoadExecutor-5175-thread-1-processing-n:127.0.0.1:40156_yv_%2Fng) [n:127.0.0.1:40156_yv_%2Fng c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 1855657 INFO  (coreLoadExecutor-5175-thread-1-processing-n:127.0.0.1:40156_yv_%2Fng) [n:127.0.0.1:40156_yv_%2Fng c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=1361902405, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
   [junit4]   2> 1855657 INFO  (coreLoadExecutor-5175-thread-1-processing-n:127.0.0.1:40156_yv_%2Fng) [n:127.0.0.1:40156_yv_%2Fng c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(NRTCachingDirectory(RAMDirectory@77689c4e lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@61f51f70; maxCacheMB=0.62109375 maxMergeSizeMB=0.955078125)),segFN=segments_1,generation=1}
   [junit4]   2> 1855658 INFO  (coreLoadExecutor-5175-thread-1-processing-n:127.0.0.1:40156_yv_%2Fng) [n:127.0.0.1:40156_yv_%2Fng c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 1855658 INFO  (coreLoadExecutor-5175-thread-1-processing-n:127.0.0.1:40156_yv_%2Fng) [n:127.0.0.1:40156_yv_%2Fng c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@39574eba[collection1] main
   [junit4]   2> 1855658 INFO  (coreLoadExecutor-5175-thread-1-processing-n:127.0.0.1:40156_yv_%2Fng) [n:127.0.0.1:40156_yv_%2Fng c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 1855659 INFO  (coreLoadExecutor-5175-thread-1-processing-n:127.0.0.1:40156_yv_%2Fng) [n:127.0.0.1:40156_yv_%2Fng c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1855659 INFO  (coreLoadExecutor-5175-thread-1-processing-n:127.0.0.1:40156_yv_%2Fng) [n:127.0.0.1:40156_yv_%2Fng c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 1855659 INFO  (coreLoadExecutor-5175-thread-1-processing-n:127.0.0.1:40156_yv_%2Fng) [n:127.0.0.1:40156_yv_%2Fng c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1855660 INFO  (coreLoadExecutor-5175-thread-1-processing-n:127.0.0.1:40156_yv_%2Fng) [n:127.0.0.1:40156_yv_%2Fng c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 1855660 INFO  (coreLoadExecutor-5175-thread-1-processing-n:127.0.0.1:40156_yv_%2Fng) [n:127.0.0.1:40156_yv_%2Fng c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1855660 INFO  (coreLoadExecutor-5175-thread-1-processing-n:127.0.0.1:40156_yv_%2Fng) [n:127.0.0.1:40156_yv_%2Fng c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 1855660 INFO  (coreLoadExecutor-5175-thread-1-processing-n:127.0.0.1:40156_yv_%2Fng) [n:127.0.0.1:40156_yv_%2Fng c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1855661 INFO  (coreLoadExecutor-5175-thread-1-processing-n:127.0.0.1:40156_yv_%2Fng) [n:127.0.0.1:40156_yv_%2Fng c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 1855661 INFO  (coreLoadExecutor-5175-thread-1-processing-n:127.0.0.1:40156_yv_%2Fng) [n:127.0.0.1:40156_yv_%2Fng c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1855661 INFO  (searcherExecutor-5176-thread-1-processing-n:127.0.0.1:40156_yv_%2Fng x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:40156_yv_%2Fng c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@39574eba[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1855662 INFO  (coreLoadExecutor-5175-thread-1-processing-n:127.0.0.1:40156_yv_%2Fng) [n:127.0.0.1:40156_yv_%2Fng c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 1855662 INFO  (coreLoadExecutor-5175-thread-1-processing-n:127.0.0.1:40156_yv_%2Fng) [n:127.0.0.1:40156_yv_%2Fng c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 1855662 INFO  (coreLoadExecutor-5175-thread-1-processing-n:127.0.0.1:40156_yv_%2Fng) [n:127.0.0.1:40156_yv_%2Fng c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 1855662 INFO  (coreLoadExecutor-5175-thread-1-processing-n:127.0.0.1:40156_yv_%2Fng) [n:127.0.0.1:40156_yv_%2Fng c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1507108211830816768
   [junit4]   2> 1855664 INFO  (coreLoadExecutor-5175-thread-1-processing-n:127.0.0.1:40156_yv_%2Fng) [n:127.0.0.1:40156_yv_%2Fng c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 2 ms to seed version buckets with highest version 1507108211830816768
   [junit4]   2> 1855665 INFO  (coreLoadExecutor-5175-thread-1-processing-n:127.0.0.1:40156_yv_%2Fng) [n:127.0.0.1:40156_yv_%2Fng c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 1855665 INFO  (coreLoadExecutor-5175-thread-1-processing-n:127.0.0.1:40156_yv_%2Fng) [n:127.0.0.1:40156_yv_%2Fng c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 1855666 INFO  (coreZkRegister-5169-thread-1-processing-n:127.0.0.1:40156_yv_%2Fng x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:40156_yv_%2Fng c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:40156/yv_/ng collection:control_collection shard:shard1
   [junit4]   2> 1855667 INFO  (coreZkRegister-5169-thread-1-processing-n:127.0.0.1:40156_yv_%2Fng x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:40156_yv_%2Fng c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 1855670 INFO  (coreZkRegister-5169-thread-1-processing-n:127.0.0.1:40156_yv_%2Fng x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:40156_yv_%2Fng c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94194263117791237-core_node1-n_0000000000
   [junit4]   2> 1855671 INFO  (coreZkRegister-5169-thread-1-processing-n:127.0.0.1:40156_yv_%2Fng x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:40156_yv_%2Fng c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 1855672 INFO  (zkCallback-2139-thread-1-processing-n:127.0.0.1:40156_yv_%2Fng) [n:127.0.0.1:40156_yv_%2Fng    ] o.a.s.c.DistributedQueue NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 1855673 INFO  (coreZkRegister-5169-thread-1-processing-n:127.0.0.1:40156_yv_%2Fng x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:40156_yv_%2Fng c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1855673 INFO  (coreZkRegister-5169-thread-1-processing-n:127.0.0.1:40156_yv_%2Fng x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:40156_yv_%2Fng c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1855673 INFO  (coreZkRegister-5169-thread-1-processing-n:127.0.0.1:40156_yv_%2Fng x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:40156_yv_%2Fng c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:40156/yv_/ng/collection1/
   [junit4]   2> 1855673 INFO  (coreZkRegister-5169-thread-1-processing-n:127.0.0.1:40156_yv_%2Fng x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:40156_yv_%2Fng c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 1855673 INFO  (OverseerStateUpdate-94194263117791237-127.0.0.1:40156_yv_%2Fng-n_0000000000) [n:127.0.0.1:40156_yv_%2Fng    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 1855673 INFO  (coreZkRegister-5169-thread-1-processing-n:127.0.0.1:40156_yv_%2Fng x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:40156_yv_%2Fng c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:40156/yv_/ng/collection1/ has no replicas
   [junit4]   2> 1855674 INFO  (coreZkRegister-5169-thread-1-processing-n:127.0.0.1:40156_yv_%2Fng x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:40156_yv_%2Fng c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:40156/yv_/ng/collection1/ shard1
   [junit4]   2> 1855674 INFO  (coreZkRegister-5169-thread-1-processing-n:127.0.0.1:40156_yv_%2Fng x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:40156_yv_%2Fng c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 1855677 INFO  (zkCallback-2139-thread-1-processing-n:127.0.0.1:40156_yv_%2Fng) [n:127.0.0.1:40156_yv_%2Fng    ] o.a.s.c.DistributedQueue NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 1855678 INFO  (OverseerStateUpdate-94194263117791237-127.0.0.1:40156_yv_%2Fng-n_0000000000) [n:127.0.0.1:40156_yv_%2Fng    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:40156/yv_/ng",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 1
   [junit4]   2> 1855781 INFO  (zkCallback-2139-thread-1-processing-n:127.0.0.1:40156_yv_%2Fng) [n:127.0.0.1:40156_yv_%2Fng    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1855782 INFO  (zkCallback-2139-thread-1-processing-n:127.0.0.1:40156_yv_%2Fng) [n:127.0.0.1:40156_yv_%2Fng    ] o.a.s.c.c.ZkStateReader Updated cluster state version to 2
   [junit4]   2> 1855828 INFO  (coreZkRegister-5169-thread-1-processing-n:127.0.0.1:40156_yv_%2Fng x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:40156_yv_%2Fng c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:40156/yv_/ng/collection1/ and leader is http://127.0.0.1:40156/yv_/ng/collection1/
   [junit4]   2> 1855828 INFO  (coreZkRegister-5169-thread-1-processing-n:127.0.0.1:40156_yv_%2Fng x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:40156_yv_%2Fng c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:40156/yv_/ng
   [junit4]   2> 1855829 INFO  (coreZkRegister-5169-thread-1-processing-n:127.0.0.1:40156_yv_%2Fng x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:40156_yv_%2Fng c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1855829 INFO  (coreZkRegister-5169-thread-1-processing-n:127.0.0.1:40156_yv_%2Fng x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:40156_yv_%2Fng c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 1855829 INFO  (coreZkRegister-5169-thread-1-processing-n:127.0.0.1:40156_yv_%2Fng x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:40156_yv_%2Fng c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1855830 INFO  (zkCallback-2139-thread-1-processing-n:127.0.0.1:40156_yv_%2Fng) [n:127.0.0.1:40156_yv_%2Fng    ] o.a.s.c.DistributedQueue NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 1855831 INFO  (OverseerStateUpdate-94194263117791237-127.0.0.1:40156_yv_%2Fng-n_0000000000) [n:127.0.0.1:40156_yv_%2Fng    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:40156/yv_/ng",
   [junit4]   2>   "node_name":"127.0.0.1:40156_yv_%2Fng",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"} current state version: 2
   [junit4]   2> 1855832 INFO  (OverseerStateUpdate-94194263117791237-127.0.0.1:40156_yv_%2Fng-n_0000000000) [n:127.0.0.1:40156_yv_%2Fng    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:40156/yv_/ng",
   [junit4]   2>   "node_name":"127.0.0.1:40156_yv_%2Fng",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 1855934 INFO  (zkCallback-2139-thread-1-processing-n:127.0.0.1:40156_yv_%2Fng) [n:127.0.0.1:40156_yv_%2Fng    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1855936 INFO  (zkCallback-2139-thread-1-processing-n:127.0.0.1:40156_yv_%2Fng) [n:127.0.0.1:40156_yv_%2Fng    ] o.a.s.c.c.ZkStateReader Updated cluster state version to 3
   [junit4]   2> 1855937 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1855938 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1855939 INFO  (zkCallback-2141-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3e7d3d7f name:ZooKeeperConnection Watcher:127.0.0.1:37397/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1855939 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1855939 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1855940 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1855942 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 1855942 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with stateFormat=2
   [junit4]   2> 1855942 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1855943 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1855944 INFO  (zkCallback-2142-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3664117e name:ZooKeeperConnection Watcher:127.0.0.1:37397/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1855944 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1855944 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1855946 INFO  (zkCallback-2139-thread-1-processing-n:127.0.0.1:40156_yv_%2Fng) [n:127.0.0.1:40156_yv_%2Fng    ] o.a.s.c.DistributedQueue NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 1855946 INFO  (OverseerStateUpdate-94194263117791237-127.0.0.1:40156_yv_%2Fng-n_0000000000) [n:127.0.0.1:40156_yv_%2Fng    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"create",
   [junit4]   2>   "name":"collection1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "stateFormat":"2"} current state version: 3
   [junit4]   2> 1855947 INFO  (OverseerStateUpdate-94194263117791237-127.0.0.1:40156_yv_%2Fng-n_0000000000) [n:127.0.0.1:40156_yv_%2Fng    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 1856050 INFO  (OverseerStateUpdate-94194263117791237-127.0.0.1:40156_yv_%2Fng-n_0000000000) [n:127.0.0.1:40156_yv_%2Fng    ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/collection1/state.json
   [junit4]   2> 1856051 INFO  (zkCallback-2141-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1856051 INFO  (zkCallback-2139-thread-1-processing-n:127.0.0.1:40156_yv_%2Fng) [n:127.0.0.1:40156_yv_%2Fng    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1856054 INFO  (zkCallback-2141-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated cluster state version to 4
   [junit4]   2> 1856054 INFO  (zkCallback-2139-thread-1-processing-n:127.0.0.1:40156_yv_%2Fng) [n:127.0.0.1:40156_yv_%2Fng    ] o.a.s.c.c.ZkStateReader Updated cluster state version to 4
   [junit4]   2> 1856292 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_B739B805AAD08D6E-001/shard-1-001/cores/collection1
   [junit4]   2> 1856293 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_B739B805AAD08D6E-001/shard-1-001
   [junit4]   2> 1856294 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.e.j.s.Server jetty-9.2.11.v20150529
   [junit4]   2> 1856296 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@6a738525{/yv_/ng,null,AVAILABLE}
   [junit4]   2> 1856297 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.e.j.s.ServerConnector Started ServerConnector@191aaf12{HTTP/1.1}{127.0.0.1:47041}
   [junit4]   2> 1856297 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.e.j.s.Server Started @1860109ms
   [junit4]   2> 1856297 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_B739B805AAD08D6E-001/tempDir-001/jetty1, solrconfig=solrconfig.xml, hostContext=/yv_/ng, hostPort=47041, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_B739B805AAD08D6E-001/shard-1-001/cores}
   [junit4]   2> 1856297 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@4e0e2f2a
   [junit4]   2> 1856297 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_B739B805AAD08D6E-001/shard-1-001/'
   [junit4]   2> 1856315 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_B739B805AAD08D6E-001/shard-1-001/solr.xml
   [junit4]   2> 1856321 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_B739B805AAD08D6E-001/shard-1-001/cores
   [junit4]   2> 1856321 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.c.CoreContainer New CoreContainer 1450657364
   [junit4]   2> 1856321 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_B739B805AAD08D6E-001/shard-1-001/]
   [junit4]   2> 1856321 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_B739B805AAD08D6E-001/shard-1-001/lib
   [junit4]   2> 1856322 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_B739B805AAD08D6E-001/shard-1-001/lib).
   [junit4]   2> 1856329 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1856331 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1856331 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1856332 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1856332 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:37397/solr
   [junit4]   2> 1856332 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1856332 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1856333 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1856334 INFO  (zkCallback-2144-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5490dc17 name:ZooKeeperConnection Watcher:127.0.0.1:37397 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1856334 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1856335 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1856336 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [n:127.0.0.1:47041_yv_%2Fng    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1856337 INFO  (zkCallback-2145-thread-1-processing-n:127.0.0.1:47041_yv_%2Fng) [n:127.0.0.1:47041_yv_%2Fng    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6558c87e name:ZooKeeperConnection Watcher:127.0.0.1:37397/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1856337 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [n:127.0.0.1:47041_yv_%2Fng    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1856341 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [n:127.0.0.1:47041_yv_%2Fng    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1857346 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [n:127.0.0.1:47041_yv_%2Fng    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:47041_yv_%2Fng
   [junit4]   2> 1857347 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B739B805AAD08D6E]) [n:127.0.0.1:47041_yv_%2Fng    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:47041_yv_%2Fng
   [j

[...truncated too long message...]

s/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/apacheds-protocol-kerberos-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/apacheds-protocol-ldap-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/apacheds-protocol-shared-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/apacheds-xdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/api-all-1.0.0-M20.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/bcprov-jdk15-1.45.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/cglib-nodep-2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/commons-collections-3.2.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/commons-math3-3.4.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/easymock-3.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/ehcache-core-2.4.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/hadoop-common-2.6.0-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/hadoop-hdfs-2.6.0-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/hadoop-minikdc-2.6.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/jackson-annotations-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/jackson-databind-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/jcl-over-slf4j-1.7.7.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/jersey-core-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/jersey-server-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/jetty-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/jetty-util-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/mina-core-2.0.0-M5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/objenesis-1.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/presto-parser-0.108.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/slice-0.10.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/build/analysis/icu/lucene-analyzers-icu-6.0.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/contrib/solr-analysis-extras/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/contrib/analysis-extras/lib/icu4j-54.1.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-launcher.jar:/x1/jenkins/.ant/lib/ivy-2.3.0.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-antlr.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-regexp.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-resolver.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-net.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jmf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-oro.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-xalan2.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-logging.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jsch.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit4.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-javamail.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jai.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jdepend.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bsf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-swing.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-netrexx.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bcel.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-testutil.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-log4j.jar:/x1/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.8/lib/tools.jar:/x1/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.1.13.jar com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/temp/junit4-J1-20150719_024854_146.events @/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/temp/junit4-J1-20150719_024854_146.suites
   [junit4] ERROR: JVM J1 ended with an exception: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1504)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:133)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:964)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:961)
   [junit4] 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4] 	at java.lang.Thread.run(Thread.java:745)

BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:533: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:474: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:61: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/extra-targets.xml:39: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build.xml:230: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/common-build.xml:513: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/common-build.xml:1426: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/common-build.xml:981: At least one slave process threw an exception, first: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.

Total time: 427 minutes 44 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
No prior successful build to compare, so performing full copy of artifacts
ERROR: Failed to archive artifacts: **/*.events,heapdumps/**,**/hs_err_pid*,README.txt
java.io.IOException: Failed to extract /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/transfer of 4 files
	at hudson.FilePath.readFromTar(FilePath.java:2101)
	at hudson.FilePath.copyRecursiveTo(FilePath.java:2013)
	at jenkins.model.StandardArtifactManager.archive(StandardArtifactManager.java:61)
	at com.cloudbees.jenkins.plugins.jsync.archiver.JSyncArtifactManager.remoteSync(JSyncArtifactManager.java:107)
	at com.cloudbees.jenkins.plugins.jsync.archiver.JSyncArtifactManager.archive(JSyncArtifactManager.java:68)
	at hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:140)
	at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
	at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:756)
	at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:720)
	at hudson.model.Build$BuildExecution.post2(Build.java:182)
	at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:669)
	at hudson.model.Run.execute(Run.java:1731)
	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
	at hudson.model.ResourceController.execute(ResourceController.java:88)
	at hudson.model.Executor.run(Executor.java:232)
Caused by: java.io.IOException: unexpected EOF with 4096 bytes unread
	at hudson.org.apache.tools.tar.TarInputStream.read(TarInputStream.java:349)
	at java.io.FilterInputStream.read(FilterInputStream.java:107)
	at org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:1792)
	at org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:1769)
	at org.apache.commons.io.IOUtils.copy(IOUtils.java:1744)
	at hudson.util.IOUtils.copy(IOUtils.java:40)
	at hudson.FilePath.readFromTar(FilePath.java:2091)
	... 14 more
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-NightlyTests-trunk - Build # 743 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-trunk/743/

3 tests failed.
REGRESSION:  org.apache.solr.cloud.ChaosMonkeySafeLeaderTest.test

Error Message:
Test abandoned because suite timeout was reached.

Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
	at __randomizedtesting.SeedInfo.seed([ABB8E213C56C6A65]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.ChaosMonkeySafeLeaderTest

Error Message:
Suite timeout exceeded (>= 7200000 msec).

Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
	at __randomizedtesting.SeedInfo.seed([ABB8E213C56C6A65]:0)


FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=54499, name=collection5, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=54499, name=collection5, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:57650: collection already exists: awholynewstresscollection_collection5_1
	at __randomizedtesting.SeedInfo.seed([ABB8E213C56C6A65]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:560)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:234)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:226)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:376)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:328)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1086)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:856)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:799)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1572)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1593)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:887)




Build Log:
[...truncated 10846 lines...]
   [junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_ABB8E213C56C6A65-001/init-core-data-001
   [junit4]   2> 1587179 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[ABB8E213C56C6A65]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false)
   [junit4]   2> 1587179 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[ABB8E213C56C6A65]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   2> 1587185 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 1587205 INFO  (Thread-49192) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1587205 INFO  (Thread-49192) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 1587305 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.c.ZkTestServer start zk server on port:36125
   [junit4]   2> 1587306 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1587309 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1587312 INFO  (zkCallback-714-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@36ff82c6 name:ZooKeeperConnection Watcher:127.0.0.1:36125 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1587312 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1587313 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1587313 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 1587318 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1587324 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1587326 INFO  (zkCallback-715-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@44ef3698 name:ZooKeeperConnection Watcher:127.0.0.1:36125/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1587326 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1587327 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1587327 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 1587328 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 1587330 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 1587332 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 1587334 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 1587334 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 1587337 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 1587338 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 1587340 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1587340 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1587342 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 1587342 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 1587344 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 1587344 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 1587346 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 1587346 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 1587348 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 1587348 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 1587351 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1587351 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1587353 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1587354 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1587356 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 1587356 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 1587358 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 1587358 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 1587362 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1587363 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1587363 INFO  (zkCallback-716-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5840c577 name:ZooKeeperConnection Watcher:127.0.0.1:36125 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1587364 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1587364 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1587364 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 1587369 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1587377 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1587378 INFO  (zkCallback-717-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@bba2b8f name:ZooKeeperConnection Watcher:127.0.0.1:36125/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1587378 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1587378 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1587379 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf2/solrconfig.xml
   [junit4]   2> 1587379 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/solrconfig.xml
   [junit4]   2> 1587382 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf2/schema.xml
   [junit4]   2> 1587382 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/schema.xml
   [junit4]   2> 1587384 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf2/enumsConfig.xml
   [junit4]   2> 1587384 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/enumsConfig.xml
   [junit4]   2> 1587386 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf2/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1587387 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1587388 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf2/stopwords.txt
   [junit4]   2> 1587388 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/stopwords.txt
   [junit4]   2> 1587390 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf2/protwords.txt
   [junit4]   2> 1587391 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/protwords.txt
   [junit4]   2> 1587394 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf2/currency.xml
   [junit4]   2> 1587395 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/currency.xml
   [junit4]   2> 1587400 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf2/open-exchange-rates.json
   [junit4]   2> 1587400 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/open-exchange-rates.json
   [junit4]   2> 1587402 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf2/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1587403 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1587405 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf2/old_synonyms.txt
   [junit4]   2> 1587405 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/old_synonyms.txt
   [junit4]   2> 1587407 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf2/synonyms.txt
   [junit4]   2> 1587408 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/synonyms.txt
   [junit4]   2> 1587410 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/elevate.xml to /configs/conf2/elevate.xml
   [junit4]   2> 1587410 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/elevate.xml
   [junit4]   2> 1587832 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_ABB8E213C56C6A65-001/control-001/cores/collection1
   [junit4]   2> 1587836 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.e.j.s.Server jetty-9.2.11.v20150529
   [junit4]   2> 1587857 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@35eeeff0{/,null,AVAILABLE}
   [junit4]   2> 1587859 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.e.j.s.ServerConnector Started ServerConnector@590a0d36{HTTP/1.1}{127.0.0.1:59154}
   [junit4]   2> 1587859 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.e.j.s.Server Started @1591092ms
   [junit4]   2> 1587859 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_ABB8E213C56C6A65-001/tempDir-001/control/data, hostContext=/, hostPort=59154, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_ABB8E213C56C6A65-001/control-001/cores}
   [junit4]   2> 1587859 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@4e0e2f2a
   [junit4]   2> 1587860 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_ABB8E213C56C6A65-001/control-001/'
   [junit4]   2> 1587878 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_ABB8E213C56C6A65-001/control-001/solr.xml
   [junit4]   2> 1587885 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_ABB8E213C56C6A65-001/control-001/cores
   [junit4]   2> 1587885 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.c.CoreContainer New CoreContainer 1791339571
   [junit4]   2> 1587886 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_ABB8E213C56C6A65-001/control-001/]
   [junit4]   2> 1587886 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_ABB8E213C56C6A65-001/control-001/lib
   [junit4]   2> 1587886 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_ABB8E213C56C6A65-001/control-001/lib).
   [junit4]   2> 1587894 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1587895 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1587896 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1587896 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1587896 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:36125/solr
   [junit4]   2> 1587896 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1587896 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1587898 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1587898 INFO  (zkCallback-719-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@55dd4d3 name:ZooKeeperConnection Watcher:127.0.0.1:36125 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1587899 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1587900 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1587904 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [n:127.0.0.1:59154_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1587906 INFO  (zkCallback-720-thread-1-processing-n:127.0.0.1:59154_) [n:127.0.0.1:59154_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@26c0b1bc name:ZooKeeperConnection Watcher:127.0.0.1:36125/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1587906 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [n:127.0.0.1:59154_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1587909 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [n:127.0.0.1:59154_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 1587911 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [n:127.0.0.1:59154_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 1587913 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [n:127.0.0.1:59154_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 1587916 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [n:127.0.0.1:59154_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 1587918 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [n:127.0.0.1:59154_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 1587921 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [n:127.0.0.1:59154_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 1587922 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [n:127.0.0.1:59154_    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 1587925 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [n:127.0.0.1:59154_    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 1587926 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [n:127.0.0.1:59154_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:59154_
   [junit4]   2> 1587927 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [n:127.0.0.1:59154_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:59154_
   [junit4]   2> 1587928 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [n:127.0.0.1:59154_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 1587930 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [n:127.0.0.1:59154_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 1587932 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [n:127.0.0.1:59154_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1587933 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [n:127.0.0.1:59154_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94188369055776773-127.0.0.1:59154_-n_0000000000
   [junit4]   2> 1587935 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [n:127.0.0.1:59154_    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:59154_
   [junit4]   2> 1587935 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [n:127.0.0.1:59154_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 1587937 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [n:127.0.0.1:59154_    ] o.a.s.c.Overseer Overseer (id=94188369055776773-127.0.0.1:59154_-n_0000000000) starting
   [junit4]   2> 1587939 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [n:127.0.0.1:59154_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 1587946 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [n:127.0.0.1:59154_    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 1587947 INFO  (OverseerCollectionProcessor-94188369055776773-127.0.0.1:59154_-n_0000000000) [n:127.0.0.1:59154_    ] o.a.s.c.OverseerCollectionProcessor Process current queue of collection creations
   [junit4]   2> 1587948 INFO  (OverseerStateUpdate-94188369055776773-127.0.0.1:59154_-n_0000000000) [n:127.0.0.1:59154_    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 1587956 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [n:127.0.0.1:59154_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1587960 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [n:127.0.0.1:59154_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1587961 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [n:127.0.0.1:59154_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1587961 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [n:127.0.0.1:59154_    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_ABB8E213C56C6A65-001/control-001/cores
   [junit4]   2> 1587962 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [n:127.0.0.1:59154_    ] o.a.s.c.SolrCore Created CoreDescriptor: {name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, loadOnStartup=true, configSetProperties=configsetprops.json, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_ABB8E213C56C6A65-001/control-001/cores/collection1, collection=control_collection, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_ABB8E213C56C6A65-001/control-001/cores/collection1/, coreNodeName=, dataDir=data/, shard=}
   [junit4]   2> 1587962 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [n:127.0.0.1:59154_    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_ABB8E213C56C6A65-001/control-001/cores/collection1/
   [junit4]   2> 1587963 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [n:127.0.0.1:59154_    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 1587964 INFO  (coreLoadExecutor-3736-thread-1-processing-n:127.0.0.1:59154_) [n:127.0.0.1:59154_ c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 1587964 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [n:127.0.0.1:59154_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2
   [junit4]   2> 1587964 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [n:127.0.0.1:59154_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1587964 INFO  (coreLoadExecutor-3736-thread-1-processing-n:127.0.0.1:59154_) [n:127.0.0.1:59154_ c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1587966 INFO  (zkCallback-720-thread-1-processing-n:127.0.0.1:59154_) [n:127.0.0.1:59154_    ] o.a.s.c.DistributedQueue NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 1587966 INFO  (coreLoadExecutor-3736-thread-1-processing-n:127.0.0.1:59154_) [n:127.0.0.1:59154_ c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 1587966 INFO  (OverseerStateUpdate-94188369055776773-127.0.0.1:59154_-n_0000000000) [n:127.0.0.1:59154_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:59154",
   [junit4]   2>   "node_name":"127.0.0.1:59154_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 1587967 INFO  (OverseerStateUpdate-94188369055776773-127.0.0.1:59154_-n_0000000000) [n:127.0.0.1:59154_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:59154",
   [junit4]   2>   "node_name":"127.0.0.1:59154_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 1587967 INFO  (OverseerStateUpdate-94188369055776773-127.0.0.1:59154_-n_0000000000) [n:127.0.0.1:59154_    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 1587967 INFO  (OverseerStateUpdate-94188369055776773-127.0.0.1:59154_-n_0000000000) [n:127.0.0.1:59154_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 1587974 INFO  (zkCallback-720-thread-1-processing-n:127.0.0.1:59154_) [n:127.0.0.1:59154_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1587976 INFO  (zkCallback-720-thread-1-processing-n:127.0.0.1:59154_) [n:127.0.0.1:59154_    ] o.a.s.c.c.ZkStateReader Updated cluster state version to 1
   [junit4]   2> 1588966 INFO  (coreLoadExecutor-3736-thread-1-processing-n:127.0.0.1:59154_) [n:127.0.0.1:59154_ c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1588966 INFO  (coreLoadExecutor-3736-thread-1-processing-n:127.0.0.1:59154_) [n:127.0.0.1:59154_ c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 1588967 INFO  (coreLoadExecutor-3736-thread-1-processing-n:127.0.0.1:59154_) [n:127.0.0.1:59154_ c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 1588967 INFO  (coreLoadExecutor-3736-thread-1-processing-n:127.0.0.1:59154_) [n:127.0.0.1:59154_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 1588968 INFO  (coreLoadExecutor-3736-thread-1-processing-n:127.0.0.1:59154_) [n:127.0.0.1:59154_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1588968 INFO  (coreLoadExecutor-3736-thread-1-processing-n:127.0.0.1:59154_) [n:127.0.0.1:59154_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_ABB8E213C56C6A65-001/control-001/cores/collection1/'
   [junit4]   2> 1588986 INFO  (coreLoadExecutor-3736-thread-1-processing-n:127.0.0.1:59154_) [n:127.0.0.1:59154_ c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 1588992 INFO  (coreLoadExecutor-3736-thread-1-processing-n:127.0.0.1:59154_) [n:127.0.0.1:59154_ c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 1589011 INFO  (coreLoadExecutor-3736-thread-1-processing-n:127.0.0.1:59154_) [n:127.0.0.1:59154_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.0.0
   [junit4]   2> 1589019 INFO  (coreLoadExecutor-3736-thread-1-processing-n:127.0.0.1:59154_) [n:127.0.0.1:59154_ c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1589020 INFO  (coreLoadExecutor-3736-thread-1-processing-n:127.0.0.1:59154_) [n:127.0.0.1:59154_ c:control_collection   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 1589025 INFO  (coreLoadExecutor-3736-thread-1-processing-n:127.0.0.1:59154_) [n:127.0.0.1:59154_ c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1589133 INFO  (coreLoadExecutor-3736-thread-1-processing-n:127.0.0.1:59154_) [n:127.0.0.1:59154_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 1589139 INFO  (coreLoadExecutor-3736-thread-1-processing-n:127.0.0.1:59154_) [n:127.0.0.1:59154_ c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 1589140 INFO  (coreLoadExecutor-3736-thread-1-processing-n:127.0.0.1:59154_) [n:127.0.0.1:59154_ c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 1589151 INFO  (coreLoadExecutor-3736-thread-1-processing-n:127.0.0.1:59154_) [n:127.0.0.1:59154_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1589153 INFO  (coreLoadExecutor-3736-thread-1-processing-n:127.0.0.1:59154_) [n:127.0.0.1:59154_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1589156 INFO  (coreLoadExecutor-3736-thread-1-processing-n:127.0.0.1:59154_) [n:127.0.0.1:59154_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1589157 WARN  (coreLoadExecutor-3736-thread-1-processing-n:127.0.0.1:59154_) [n:127.0.0.1:59154_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 1589157 WARN  (coreLoadExecutor-3736-thread-1-processing-n:127.0.0.1:59154_) [n:127.0.0.1:59154_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 1589157 INFO  (coreLoadExecutor-3736-thread-1-processing-n:127.0.0.1:59154_) [n:127.0.0.1:59154_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1589158 WARN  (coreLoadExecutor-3736-thread-1-processing-n:127.0.0.1:59154_) [n:127.0.0.1:59154_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 1589158 WARN  (coreLoadExecutor-3736-thread-1-processing-n:127.0.0.1:59154_) [n:127.0.0.1:59154_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 1589159 INFO  (coreLoadExecutor-3736-thread-1-processing-n:127.0.0.1:59154_) [n:127.0.0.1:59154_ c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties
   [junit4]   2> org.apache.solr.core.SolrResourceNotFoundException: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2
   [junit4]   2> 	at org.apache.solr.cloud.ZkSolrResourceLoader.openResource(ZkSolrResourceLoader.java:99)
   [junit4]   2> 	at org.apache.solr.core.ConfigSetProperties.readFromResourceLoader(ConfigSetProperties.java:49)
   [junit4]   2> 	at org.apache.solr.core.ConfigSetService.createConfigSetProperties(ConfigSetService.java:114)
   [junit4]   2> 	at org.apache.solr.core.ConfigSetService.getConfig(ConfigSetService.java:76)
   [junit4]   2> 	at org.apache.solr.core.CoreContainer.create(CoreContainer.java:668)
   [junit4]   2> 	at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:397)
   [junit4]   2> 	at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:388)
   [junit4]   2> 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
   [junit4]   2> 	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:156)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 1589160 INFO  (coreLoadExecutor-3736-thread-1-processing-n:127.0.0.1:59154_) [n:127.0.0.1:59154_ c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 1589160 INFO  (coreLoadExecutor-3736-thread-1-processing-n:127.0.0.1:59154_) [n:127.0.0.1:59154_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 1589160 INFO  (coreLoadExecutor-3736-thread-1-processing-n:127.0.0.1:59154_) [n:127.0.0.1:59154_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_ABB8E213C56C6A65-001/control-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 1589160 INFO  (coreLoadExecutor-3736-thread-1-processing-n:127.0.0.1:59154_) [n:127.0.0.1:59154_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@5a1adc91
   [junit4]   2> 1589161 INFO  (coreLoadExecutor-3736-thread-1-processing-n:127.0.0.1:59154_) [n:127.0.0.1:59154_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_ABB8E213C56C6A65-001/control-001/cores/collection1/data
   [junit4]   2> 1589161 INFO  (coreLoadExecutor-3736-thread-1-processing-n:127.0.0.1:59154_) [n:127.0.0.1:59154_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_ABB8E213C56C6A65-001/control-001/cores/collection1/data/index/
   [junit4]   2> 1589161 WARN  (coreLoadExecutor-3736-thread-1-processing-n:127.0.0.1:59154_) [n:127.0.0.1:59154_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_ABB8E213C56C6A65-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 1589161 INFO  (coreLoadExecutor-3736-thread-1-processing-n:127.0.0.1:59154_) [n:127.0.0.1:59154_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_ABB8E213C56C6A65-001/control-001/cores/collection1/data/index
   [junit4]   2> 1589162 INFO  (coreLoadExecutor-3736-thread-1-processing-n:127.0.0.1:59154_) [n:127.0.0.1:59154_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=2037095973, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
   [junit4]   2> 1589162 INFO  (coreLoadExecutor-3736-thread-1-processing-n:127.0.0.1:59154_) [n:127.0.0.1:59154_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=RawDirectoryWrapper(RAMDirectory@400dd32b lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2048d07c),segFN=segments_1,generation=1}
   [junit4]   2> 1589163 INFO  (coreLoadExecutor-3736-thread-1-processing-n:127.0.0.1:59154_) [n:127.0.0.1:59154_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 1589163 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:59154_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_ABB8E213C56C6A65-001/control-001/cores/collection1/data/
   [junit4]   2> 1589169 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:59154_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_ABB8E213C56C6A65-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 1589166 INFO  (coreLoadExecutor-3736-thread-1-processing-n:127.0.0.1:59154_) [n:127.0.0.1:59154_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 1589172 INFO  (coreLoadExecutor-3736-thread-1-processing-n:127.0.0.1:59154_) [n:127.0.0.1:59154_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 1589172 INFO  (coreLoadExecutor-3736-thread-1-processing-n:127.0.0.1:59154_) [n:127.0.0.1:59154_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 1589172 INFO  (coreLoadExecutor-3736-thread-1-processing-n:127.0.0.1:59154_) [n:127.0.0.1:59154_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1589172 INFO  (coreLoadExecutor-3736-thread-1-processing-n:127.0.0.1:59154_) [n:127.0.0.1:59154_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1589172 INFO  (coreLoadExecutor-3736-thread-1-processing-n:127.0.0.1:59154_) [n:127.0.0.1:59154_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 1589173 INFO  (coreLoadExecutor-3736-thread-1-processing-n:127.0.0.1:59154_) [n:127.0.0.1:59154_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1589173 INFO  (coreLoadExecutor-3736-thread-1-processing-n:127.0.0.1:59154_) [n:127.0.0.1:59154_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1589173 INFO  (coreLoadExecutor-3736-thread-1-processing-n:127.0.0.1:59154_) [n:127.0.0.1:59154_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 1589175 INFO  (coreLoadExecutor-3736-thread-1-processing-n:127.0.0.1:59154_) [n:127.0.0.1:59154_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1589176 INFO  (coreLoadExecutor-3736-thread-1-processing-n:127.0.0.1:59154_) [n:127.0.0.1:59154_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1589177 INFO  (coreLoadExecutor-3736-thread-1-processing-n:127.0.0.1:59154_) [n:127.0.0.1:59154_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1589178 INFO  (coreLoadExecutor-3736-thread-1-processing-n:127.0.0.1:59154_) [n:127.0.0.1:59154_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1589186 INFO  (coreLoadExecutor-3736-thread-1-processing-n:127.0.0.1:59154_) [n:127.0.0.1:59154_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/mbeans,standard,/update/csv,/update/json/docs,/admin/luke,/admin/segments,/get,/admin/system,/replication,/admin/properties,/config,/schema,/admin/plugins,/admin/logging,/update/json,/admin/threads,/admin/ping,/update,/admin/file
   [junit4]   2> 1589187 INFO  (coreLoadExecutor-3736-thread-1-processing-n:127.0.0.1:59154_) [n:127.0.0.1:59154_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 1589188 INFO  (coreLoadExecutor-3736-thread-1-processing-n:127.0.0.1:59154_) [n:127.0.0.1:59154_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1589189 INFO  (coreLoadExecutor-3736-thread-1-processing-n:127.0.0.1:59154_) [n:127.0.0.1:59154_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1589190 INFO  (coreLoadExecutor-3736-thread-1-processing-n:127.0.0.1:59154_) [n:127.0.0.1:59154_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 1589190 INFO  (coreLoadExecutor-3736-thread-1-processing-n:127.0.0.1:59154_) [n:127.0.0.1:59154_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 1589190 INFO  (coreLoadExecutor-3736-thread-1-processing-n:127.0.0.1:59154_) [n:127.0.0.1:59154_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=11, maxMergeAtOnceExplicit=43, maxMergedSegmentMB=30.46484375, floorSegmentMB=0.7255859375, forceMergeDeletesPctAllowed=12.953108642077433, segmentsPerTier=47.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 1589191 INFO  (coreLoadExecutor-3736-thread-1-processing-n:127.0.0.1:59154_) [n:127.0.0.1:59154_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=RawDirectoryWrapper(RAMDirectory@400dd32b lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2048d07c),segFN=segments_1,generation=1}
   [junit4]   2> 1589191 INFO  (coreLoadExecutor-3736-thread-1-processing-n:127.0.0.1:59154_) [n:127.0.0.1:59154_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 1589191 INFO  (coreLoadExecutor-3736-thread-1-processing-n:127.0.0.1:59154_) [n:127.0.0.1:59154_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@4c75aac4[collection1] main
   [junit4]   2> 1589191 INFO  (coreLoadExecutor-3736-thread-1-processing-n:127.0.0.1:59154_) [n:127.0.0.1:59154_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 1589192 INFO  (coreLoadExecutor-3736-thread-1-processing-n:127.0.0.1:59154_) [n:127.0.0.1:59154_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1589192 INFO  (coreLoadExecutor-3736-thread-1-processing-n:127.0.0.1:59154_) [n:127.0.0.1:59154_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 1589193 INFO  (coreLoadExecutor-3736-thread-1-processing-n:127.0.0.1:59154_) [n:127.0.0.1:59154_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1589193 INFO  (coreLoadExecutor-3736-thread-1-processing-n:127.0.0.1:59154_) [n:127.0.0.1:59154_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 1589193 INFO  (coreLoadExecutor-3736-thread-1-processing-n:127.0.0.1:59154_) [n:127.0.0.1:59154_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1589193 INFO  (coreLoadExecutor-3736-thread-1-processing-n:127.0.0.1:59154_) [n:127.0.0.1:59154_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 1589194 INFO  (coreLoadExecutor-3736-thread-1-processing-n:127.0.0.1:59154_) [n:127.0.0.1:59154_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1589194 INFO  (coreLoadExecutor-3736-thread-1-processing-n:127.0.0.1:59154_) [n:127.0.0.1:59154_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 1589194 INFO  (coreLoadExecutor-3736-thread-1-processing-n:127.0.0.1:59154_) [n:127.0.0.1:59154_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1589195 INFO  (searcherExecutor-3737-thread-1-processing-n:127.0.0.1:59154_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:59154_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@4c75aac4[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1589195 INFO  (coreLoadExecutor-3736-thread-1-processing-n:127.0.0.1:59154_) [n:127.0.0.1:59154_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 1589195 INFO  (coreLoadExecutor-3736-thread-1-processing-n:127.0.0.1:59154_) [n:127.0.0.1:59154_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 1589195 INFO  (coreLoadExecutor-3736-thread-1-processing-n:127.0.0.1:59154_) [n:127.0.0.1:59154_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 1589195 INFO  (coreLoadExecutor-3736-thread-1-processing-n:127.0.0.1:59154_) [n:127.0.0.1:59154_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1507013906972803072
   [junit4]   2> 1589197 INFO  (coreLoadExecutor-3736-thread-1-processing-n:127.0.0.1:59154_) [n:127.0.0.1:59154_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 2 ms to seed version buckets with highest version 1507013906972803072
   [junit4]   2> 1589198 INFO  (coreLoadExecutor-3736-thread-1-processing-n:127.0.0.1:59154_) [n:127.0.0.1:59154_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 1589198 INFO  (coreLoadExecutor-3736-thread-1-processing-n:127.0.0.1:59154_) [n:127.0.0.1:59154_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 1589199 INFO  (coreZkRegister-3730-thread-1-processing-n:127.0.0.1:59154_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:59154_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:59154 collection:control_collection shard:shard1
   [junit4]   2> 1589200 INFO  (coreZkRegister-3730-thread-1-processing-n:127.0.0.1:59154_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:59154_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 1589207 INFO  (coreZkRegister-3730-thread-1-processing-n:127.0.0.1:59154_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:59154_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94188369055776773-core_node1-n_0000000000
   [junit4]   2> 1589208 INFO  (coreZkRegister-3730-thread-1-processing-n:127.0.0.1:59154_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:59154_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 1589209 INFO  (zkCallback-720-thread-1-processing-n:127.0.0.1:59154_) [n:127.0.0.1:59154_    ] o.a.s.c.DistributedQueue NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 1589209 INFO  (OverseerStateUpdate-94188369055776773-127.0.0.1:59154_-n_0000000000) [n:127.0.0.1:59154_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 1589211 INFO  (coreZkRegister-3730-thread-1-processing-n:127.0.0.1:59154_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:59154_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1589211 INFO  (coreZkRegister-3730-thread-1-processing-n:127.0.0.1:59154_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:59154_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1589211 INFO  (coreZkRegister-3730-thread-1-processing-n:127.0.0.1:59154_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:59154_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:59154/collection1/
   [junit4]   2> 1589211 INFO  (coreZkRegister-3730-thread-1-processing-n:127.0.0.1:59154_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:59154_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 1589212 INFO  (coreZkRegister-3730-thread-1-processing-n:127.0.0.1:59154_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:59154_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:59154/collection1/ has no replicas
   [junit4]   2> 1589212 INFO  (coreZkRegister-3730-thread-1-processing-n:127.0.0.1:59154_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:59154_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:59154/collection1/ shard1
   [junit4]   2> 1589212 INFO  (coreZkRegister-3730-thread-1-processing-n:127.0.0.1:59154_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:59154_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 1589215 INFO  (zkCallback-720-thread-1-processing-n:127.0.0.1:59154_) [n:127.0.0.1:59154_    ] o.a.s.c.DistributedQueue NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 1589215 INFO  (OverseerStateUpdate-94188369055776773-127.0.0.1:59154_-n_0000000000) [n:127.0.0.1:59154_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:59154",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 1
   [junit4]   2> 1589318 INFO  (zkCallback-720-thread-1-processing-n:127.0.0.1:59154_) [n:127.0.0.1:59154_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1589319 INFO  (zkCallback-720-thread-1-processing-n:127.0.0.1:59154_) [n:127.0.0.1:59154_    ] o.a.s.c.c.ZkStateReader Updated cluster state version to 2
   [junit4]   2> 1589368 INFO  (coreZkRegister-3730-thread-1-processing-n:127.0.0.1:59154_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:59154_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:59154/collection1/ and leader is http://127.0.0.1:59154/collection1/
   [junit4]   2> 1589368 INFO  (coreZkRegister-3730-thread-1-processing-n:127.0.0.1:59154_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:59154_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:59154
   [junit4]   2> 1589368 INFO  (coreZkRegister-3730-thread-1-processing-n:127.0.0.1:59154_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:59154_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1589368 INFO  (coreZkRegister-3730-thread-1-processing-n:127.0.0.1:59154_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:59154_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 1589369 INFO  (coreZkRegister-3730-thread-1-processing-n:127.0.0.1:59154_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:59154_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1589370 INFO  (zkCallback-720-thread-1-processing-n:127.0.0.1:59154_) [n:127.0.0.1:59154_    ] o.a.s.c.DistributedQueue NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 1589370 INFO  (OverseerStateUpdate-94188369055776773-127.0.0.1:59154_-n_0000000000) [n:127.0.0.1:59154_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:59154",
   [junit4]   2>   "node_name":"127.0.0.1:59154_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"} current state version: 2
   [junit4]   2> 1589371 INFO  (OverseerStateUpdate-94188369055776773-127.0.0.1:59154_-n_0000000000) [n:127.0.0.1:59154_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:59154",
   [junit4]   2>   "node_name":"127.0.0.1:59154_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 1589466 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1589469 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1589470 INFO  (zkCallback-722-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@9b2b172 name:ZooKeeperConnection Watcher:127.0.0.1:36125/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1589470 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1589470 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1589470 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1589473 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 1589478 INFO  (zkCallback-720-thread-1-processing-n:127.0.0.1:59154_) [n:127.0.0.1:59154_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1589480 INFO  (zkCallback-720-thread-1-processing-n:127.0.0.1:59154_) [n:127.0.0.1:59154_    ] o.a.s.c.c.ZkStateReader Updated cluster state version to 3
   [junit4]   2> 1589480 INFO  (zkCallback-722-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1589485 INFO  (zkCallback-722-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated cluster state version to 3
   [junit4]   2> 1589910 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_ABB8E213C56C6A65-001/shard-1-001/cores/collection1
   [junit4]   2> 1589912 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_ABB8E213C56C6A65-001/shard-1-001
   [junit4]   2> 1589913 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.e.j.s.Server jetty-9.2.11.v20150529
   [junit4]   2> 1589919 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@69d7dae{/,null,AVAILABLE}
   [junit4]   2> 1589919 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.e.j.s.ServerConnector Started ServerConnector@3d53adba{HTTP/1.1}{127.0.0.1:52301}
   [junit4]   2> 1589919 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.e.j.s.Server Started @1593152ms
   [junit4]   2> 1589919 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_ABB8E213C56C6A65-001/tempDir-001/jetty1, solrconfig=solrconfig.xml, hostContext=/, hostPort=52301, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_ABB8E213C56C6A65-001/shard-1-001/cores}
   [junit4]   2> 1589920 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@4e0e2f2a
   [junit4]   2> 1589920 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_ABB8E213C56C6A65-001/shard-1-001/'
   [junit4]   2> 1589936 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_ABB8E213C56C6A65-001/shard-1-001/solr.xml
   [junit4]   2> 1589942 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_ABB8E213C56C6A65-001/shard-1-001/cores
   [junit4]   2> 1589943 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.c.CoreContainer New CoreContainer 568788228
   [junit4]   2> 1589943 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_ABB8E213C56C6A65-001/shard-1-001/]
   [junit4]   2> 1589943 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_ABB8E213C56C6A65-001/shard-1-001/lib
   [junit4]   2> 1589943 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_ABB8E213C56C6A65-001/shard-1-001/lib).
   [junit4]   2> 1589953 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1589954 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1589955 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1589955 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1589955 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:36125/solr
   [junit4]   2> 1589955 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1589955 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1589965 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1589966 INFO  (zkCallback-724-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5139775 name:ZooKeeperConnection Watcher:127.0.0.1:36125 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1589967 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1589967 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1589989 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [n:127.0.0.1:52301_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1589993 INFO  (zkCallback-725-thread-1-processing-n:127.0.0.1:52301_) [n:127.0.0.1:52301_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@39ce7944 name:ZooKeeperConnection Watcher:127.0.0.1:36125/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1589994 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [n:127.0.0.1:52301_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1590000 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [n:127.0.0.1:52301_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1591006 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [n:127.0.0.1:52301_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:52301_
   [junit4]   2> 1591007 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [n:127.0.0.1:52301_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:52301_
   [junit4]   2> 1591011 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [n:127.0.0.1:52301_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1591014 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [n:127.0.0.1:52301_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94188369055776776-127.0.0.1:52301_-n_0000000001
   [junit4]   2> 1591015 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [n:127.0.0.1:52301_    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/94188369055776773-127.0.0.1:59154_-n_0000000000 to know if I could be the leader
   [junit4]   2> 1591015 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [n:127.0.0.1:52301_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1591016 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [n:127.0.0.1:52301_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1591017 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [n:127.0.0.1:52301_    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_ABB8E213C56C6A65-001/shard-1-001/cores
   [junit4]   2> 1591019 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [n:127.0.0.1:52301_    ] o.a.s.c.SolrCore Created CoreDescriptor: {name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, loadOnStartup=true, configSetProperties=configsetprops.json, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_ABB8E213C56C6A65-001/shard-1-001/cores/collection1, collection=collection1, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_ABB8E213C56C6A65-001/shard-1-001/cores/collection1/, coreNodeName=, dataDir=data/, shard=}
   [junit4]   2> 1591019 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [n:127.0.0.1:52301_    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_ABB8E213C56C6A65-001/shard-1-001/cores/collection1/
   [junit4]   2> 1591019 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [n:127.0.0.1:52301_    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 1591035 INFO  (coreLoadExecutor-3747-thread-1-processing-n:127.0.0.1:52301_) [n:127.0.0.1:52301_ c:collection1   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 1591035 INFO  (coreLoadExecutor-3747-thread-1-processing-n:127.0.0.1:52301_) [n:127.0.0.1:52301_ c:collection1   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1591036 INFO  (zkCallback-720-thread-1-processing-n:127.0.0.1:59154_) [n:127.0.0.1:59154_    ] o.a.s.c.DistributedQueue NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 1591036 INFO  (coreLoadExecutor-3747-thread-1-processing-n:127.0.0.1:52301_) [n:127.0.0.1:52301_ c:collection1   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 1591037 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [n:127.0.0.1:52301_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2
   [junit4]   2> 1591038 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[ABB8E213C56C6A65]) [n:127.0.0.1:52301_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1591038 INFO  (OverseerStateUpdate-94188369055776773-127.0.0.1:59154_-n_0000000000) [n:127.0.0.1:59154_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:52301",
   [junit4]   2>   "node_name":"127.0.0.1:52301_",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "operation":"state"} current state version: 3
   [junit4]   2> 1591038 INFO  (OverseerStateUpdate-94188369055776773-127.0.0.1:59154_-n_0000000000) [n:127.0.0.1:59154_    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:52301",
   [junit4]   2>   "node_name":"127.0.0.1:52301_",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 1591039 INFO  (OverseerStateUpdate-94188369055776773-127.0.0.1:59154_-n_0000000000) [n:127.0.0.1:59154_    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 1591039 INFO  (OverseerStateUpdate-94188369055776773-127.0.0.1:59154_-n_0000000000) [n:127.0.0.1:59154_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2
   [junit4]   2> 1591040 INFO  (zkCallback-720-thread-1-processing-n:127.0.0.1:59154_) [n:127.0.0.1:59154_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1591040 INFO  (zkCallback-725-thread-1-processing-n:127.0.0.1:52301_) [n:127.0.0.1:52301_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1591040 INFO  (zkCallback-722-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1591042 INFO  (zkCallback-722-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated cluster state version to 4
   [junit4]   2> 1591042 INFO  (zkCallback-720-thread-1-processing-n:127.0.0.1:59154_) [n:127.0.0.1:59154_    ] o.a.s.c.c.ZkStateReader Updated cluster state version to 4
   [junit4]   2> 1591042 INFO  (zkCallback-725-thread-1-processing-n:127.0.0.1:52301_) [n:127.0.0.1:52301_    ] o.a.s.c.c.ZkStateReader Updated cluster state version to 4
   [junit4]   2> 1592037 INFO  (coreLoadExecutor-3747-thread-1-processing-n:127.0.0.1:52301_) [n:127.0.0.1:52301_ c:collection1   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1592037 INFO  (coreLoadExecutor-3747-thread-1-processing-n:127.0.0.1:52301_) [n:127.0.0.1:52301_ c:collection1   x:collection1] o.a.s.c.ZkController Check for collection zkNode:collection1
   [junit4]   2> 1592038 INFO  (coreLoadExecutor-3747-thread-1-processing-n:127.0.0.1:52301_) [n:127.0.0.1:52301_ c:collection1   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 1592038 INFO  (coreLoadExecutor-3747-thread-1-processing-n:127.0.0.1:52301_) [n:127.0.0.1:52301_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/collection1
   [junit4]   2> 1592039 INFO  (coreLoadExecutor-3747-thread-1-processing-n:127.0.0.1:52301_) [n:127.0.0.1:52301_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1592039 INFO  (coreLoadExecutor-3747-thread-1-processing-n:127.0.0.1:52301_) [n:127.0.0.1:52301_ c:collection1   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_ABB8E213C56C6A65-001/shard-1-001/cores/collection1/'
   [junit4]   2> 1592053 INFO  (coreLoadExecutor-3747-thread-1-processing-n:127.0.0.1:52301_) [n:127.0.0.1:52301_ c:collection1   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 1592060 INFO  (coreLoadExecutor-3747-thread-1-processing-n:127.0.0.1:52301_) [n:127.0.0.1:52301_ c:collection1   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 1592063 INFO  (coreLoadExecutor-3747-thread-1-processing-n:127.0.0.1:52301_) [n:127.0.0.1:52301_ c:collection1   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.0.0
   [junit4]   2> 1592075 INFO  (coreLoadExecutor-3747-thread-1-processing-n:127.0.0.1:52301_) [n:127.0.0.1:52301_ c:collection1   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1592075 INFO  (coreLoadExecutor-3747-thread-1-processing-n:127.0.0.1:52301_) [n:127.0.0.1:52301_ c:collection1   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 1592080 INFO  (coreLoadExecutor-3747-thread-1-processing-n:127.0.0.1:52301_) [n:127.0.0.1:52301_ c:collection1   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1592191 INFO  (coreLoadExecutor-3747-thread-1-processing-n:127.0.0.1:52301_) [n:127.0.0.1:52301_ c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 1592211 INFO  (coreLoadExecutor-3747-thread-1-processing-n:127.0.0.1:52301_) [n:127.0.0.1:52301_ c:collection1   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 1592212 INFO  (coreLoadExecutor-3747-thread-1-processing-n:127.0.0.1:52301_) [n:127.0.0.1:52301_ c:collection1   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 1592220 INFO  (coreLoadExecutor-3747-thread-1-processing-n:127.0.0.1:52301_) [n:127.0.0.1:52301_ c:collection1   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1592222 INFO  (coreLoadExecutor-3747-thread-1-processing-n:127.0.0.1:52301_) [n:127.0.0.1:52301_ c:collection1   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1592224 INFO  (coreLoadExecutor-3747-thread-1-processing-n:127.0.0.1:52301_) [n:127.0.0.1:52301_ c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1592225 WARN  (coreLoadExecutor-3747-thread-1-processing-n:127.0.0.1:52301_) [n:127.0.0.1:52301_ c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 1592225 WARN  (coreLoadExecutor-3747-thread-1-processing-n:127.0.0.1:52301_) [n:127.0.0.1:52301_ c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 1592226 INFO  (coreLoadExecutor-3747-thread-1-processing-n:127.0.0.1:52301_) [n:127.0.0.1:52301_ c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1592226 WARN  (coreLoadExecutor-3747-thread-1-processing-n:127.0.0.1:52301_) [n:127.0.0.1:52301_ c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 1592226 WARN  (coreLoadExecutor-3747-thread-1-processing-n:127.0.0.1:52301_) [n:127.0.0.1:52301_ c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 1592227 INFO  (coreLoadExecutor-3747-thread-1-processing-n:127.0.0.1:52301_) [n:127.0.0.1:52301_ c:collection1   x:collection1] o.a

[...truncated too long message...]

s-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/easymock-3.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/ehcache-core-2.4.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/hadoop-common-2.6.0-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/hadoop-hdfs-2.6.0-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/hadoop-minikdc-2.6.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/jackson-annotations-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/jackson-databind-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/jcl-over-slf4j-1.7.7.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/jersey-core-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/jersey-server-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/jetty-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/jetty-util-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/mina-core-2.0.0-M5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/objenesis-1.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/presto-parser-0.108.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/slice-0.10.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/build/analysis/icu/lucene-analyzers-icu-6.0.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/contrib/solr-analysis-extras/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/contrib/analysis-extras/lib/icu4j-54.1.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-launcher.jar:/x1/jenkins/.ant/lib/ivy-2.3.0.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-antlr.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-regexp.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-resolver.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-net.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jmf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-oro.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-xalan2.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-logging.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jsch.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit4.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-javamail.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jai.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jdepend.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bsf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-swing.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-netrexx.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bcel.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-testutil.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-log4j.jar:/x1/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.8/lib/tools.jar:/x1/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.1.13.jar com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/temp/junit4-J2-20150718_015425_057.events @/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/temp/junit4-J2-20150718_015425_057.suites
   [junit4] ERROR: JVM J2 ended with an exception: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1504)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:133)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:964)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:961)
   [junit4] 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4] 	at java.lang.Thread.run(Thread.java:745)

BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:533: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:474: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:61: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/extra-targets.xml:39: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build.xml:230: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/common-build.xml:513: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/common-build.xml:1426: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/common-build.xml:981: At least one slave process threw an exception, first: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.

Total time: 444 minutes 37 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
No prior successful build to compare, so performing full copy of artifacts
ERROR: Failed to archive {README.txt=README.txt, solr/build/solr-core/test/temp/junit4-J0-20150718_015425_055.events=solr/build/solr-core/test/temp/junit4-J0-20150718_015425_055.events, solr/build/solr-core/test/temp/junit4-J1-20150718_015425_055.events=solr/build/solr-core/test/temp/junit4-J1-20150718_015425_055.events, solr/build/solr-core/test/temp/junit4-J2-20150718_015425_057.events=solr/build/solr-core/test/temp/junit4-J2-20150718_015425_057.events} due to internal error; falling back to full archiving
java.lang.IllegalArgumentException: Negative time
	at java.io.File.setLastModified(File.java:1421)
	at hudson.FilePath.readFromTar(FilePath.java:2093)
	at hudson.FilePath.copyRecursiveTo(FilePath.java:2013)
	at jenkins.model.StandardArtifactManager.archive(StandardArtifactManager.java:61)
	at com.cloudbees.jenkins.plugins.jsync.archiver.JSyncArtifactManager.remoteSync(JSyncArtifactManager.java:107)
	at com.cloudbees.jenkins.plugins.jsync.archiver.JSyncArtifactManager.archive(JSyncArtifactManager.java:68)
	at hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:140)
	at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
	at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:756)
	at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:720)
	at hudson.model.Build$BuildExecution.post2(Build.java:182)
	at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:669)
	at hudson.model.Run.execute(Run.java:1731)
	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
	at hudson.model.ResourceController.execute(ResourceController.java:88)
	at hudson.model.Executor.run(Executor.java:232)
ERROR: Publisher hudson.tasks.ArtifactArchiver aborted due to exception
java.lang.IllegalArgumentException: Negative time
	at java.io.File.setLastModified(File.java:1421)
	at hudson.FilePath.readFromTar(FilePath.java:2093)
	at hudson.FilePath.copyRecursiveTo(FilePath.java:2013)
	at jenkins.model.StandardArtifactManager.archive(StandardArtifactManager.java:61)
	at com.cloudbees.jenkins.plugins.jsync.archiver.JSyncArtifactManager.archive(JSyncArtifactManager.java:72)
	at hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:140)
	at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
	at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:756)
	at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:720)
	at hudson.model.Build$BuildExecution.post2(Build.java:182)
	at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:669)
	at hudson.model.Run.execute(Run.java:1731)
	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
	at hudson.model.ResourceController.execute(ResourceController.java:88)
	at hudson.model.Executor.run(Executor.java:232)
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-NightlyTests-trunk - Build # 742 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-trunk/742/

2 tests failed.
FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=2916, name=collection4, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=2916, name=collection4, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:33787/_je: collection already exists: awholynewstresscollection_collection4_0
	at __randomizedtesting.SeedInfo.seed([952A6D9CE6F86E0A]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:560)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:234)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:226)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:376)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:328)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1086)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:856)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:799)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1572)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1593)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:887)


FAILED:  org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=61089, name=collection4, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=61089, name=collection4, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:38745: Could not find collection : awholynewstresscollection_collection4_0
	at __randomizedtesting.SeedInfo.seed([952A6D9CE6F86E0A]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:560)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:234)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:226)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:376)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:328)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1086)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:856)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:799)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:894)




Build Log:
[...truncated 10422 lines...]
   [junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_952A6D9CE6F86E0A-001/init-core-data-001
   [junit4]   2> 868763 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[952A6D9CE6F86E0A]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false)
   [junit4]   2> 868763 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[952A6D9CE6F86E0A]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /_je/
   [junit4]   2> 868766 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 868773 INFO  (Thread-1708) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 868773 INFO  (Thread-1708) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 868870 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.c.ZkTestServer start zk server on port:53435
   [junit4]   2> 868871 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 868886 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 868902 INFO  (zkCallback-143-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@f52119e name:ZooKeeperConnection Watcher:127.0.0.1:53435 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 868903 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 868903 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 868903 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 868910 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 868919 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 868922 INFO  (zkCallback-144-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2168bef3 name:ZooKeeperConnection Watcher:127.0.0.1:53435/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 868922 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 868922 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 868922 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 868924 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 868926 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 868927 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 868928 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 868929 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 868931 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 868931 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 868932 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 868933 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 868934 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 868934 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 868935 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 868936 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 868937 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 868937 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 868939 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 868939 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 868940 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 868941 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 868942 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 868942 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 868944 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 868944 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 868945 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 868946 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 868948 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 868950 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 868959 INFO  (zkCallback-145-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@39b8bcc9 name:ZooKeeperConnection Watcher:127.0.0.1:53435 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 868959 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 868959 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 868959 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 868962 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 868966 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 868975 INFO  (zkCallback-146-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6b6d3b57 name:ZooKeeperConnection Watcher:127.0.0.1:53435/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 868975 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 868976 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 868976 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf2/solrconfig.xml
   [junit4]   2> 868976 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/solrconfig.xml
   [junit4]   2> 868979 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf2/schema.xml
   [junit4]   2> 868980 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/schema.xml
   [junit4]   2> 868981 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf2/enumsConfig.xml
   [junit4]   2> 868981 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/enumsConfig.xml
   [junit4]   2> 868983 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf2/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 868983 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 868985 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf2/stopwords.txt
   [junit4]   2> 868985 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/stopwords.txt
   [junit4]   2> 868990 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf2/protwords.txt
   [junit4]   2> 868990 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/protwords.txt
   [junit4]   2> 868992 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf2/currency.xml
   [junit4]   2> 868992 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/currency.xml
   [junit4]   2> 868993 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf2/open-exchange-rates.json
   [junit4]   2> 868994 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/open-exchange-rates.json
   [junit4]   2> 868995 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf2/mapping-ISOLatin1Accent.txt
   [junit4]   2> 868996 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/mapping-ISOLatin1Accent.txt
   [junit4]   2> 868997 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf2/old_synonyms.txt
   [junit4]   2> 868997 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/old_synonyms.txt
   [junit4]   2> 868999 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf2/synonyms.txt
   [junit4]   2> 868999 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/synonyms.txt
   [junit4]   2> 869003 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/elevate.xml to /configs/conf2/elevate.xml
   [junit4]   2> 869003 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/elevate.xml
   [junit4]   2> 869324 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_952A6D9CE6F86E0A-001/control-001/cores/collection1
   [junit4]   2> 869326 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.e.j.s.Server jetty-9.2.11.v20150529
   [junit4]   2> 869328 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5fd2679a{/_je,null,AVAILABLE}
   [junit4]   2> 869336 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.e.j.s.ServerConnector Started ServerConnector@232818b{HTTP/1.1}{127.0.0.1:44153}
   [junit4]   2> 869336 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.e.j.s.Server Started @872541ms
   [junit4]   2> 869336 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_952A6D9CE6F86E0A-001/tempDir-001/control/data, hostContext=/_je, hostPort=44153, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_952A6D9CE6F86E0A-001/control-001/cores}
   [junit4]   2> 869339 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@4e0e2f2a
   [junit4]   2> 869339 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_952A6D9CE6F86E0A-001/control-001/'
   [junit4]   2> 869355 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_952A6D9CE6F86E0A-001/control-001/solr.xml
   [junit4]   2> 869361 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_952A6D9CE6F86E0A-001/control-001/cores
   [junit4]   2> 869361 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.c.CoreContainer New CoreContainer 369019635
   [junit4]   2> 869361 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_952A6D9CE6F86E0A-001/control-001/]
   [junit4]   2> 869361 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_952A6D9CE6F86E0A-001/control-001/lib
   [junit4]   2> 869362 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_952A6D9CE6F86E0A-001/control-001/lib).
   [junit4]   2> 869369 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 869371 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 869371 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 869371 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 869371 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:53435/solr
   [junit4]   2> 869372 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 869372 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 869373 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 869375 INFO  (zkCallback-148-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@78317a06 name:ZooKeeperConnection Watcher:127.0.0.1:53435 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 869378 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 869379 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 869383 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [n:127.0.0.1:44153__je    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 869389 INFO  (zkCallback-149-thread-1-processing-n:127.0.0.1:44153__je) [n:127.0.0.1:44153__je    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@d01c485 name:ZooKeeperConnection Watcher:127.0.0.1:53435/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 869390 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [n:127.0.0.1:44153__je    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 869392 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [n:127.0.0.1:44153__je    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 869394 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [n:127.0.0.1:44153__je    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 869396 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [n:127.0.0.1:44153__je    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 869398 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [n:127.0.0.1:44153__je    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 869401 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [n:127.0.0.1:44153__je    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 869403 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [n:127.0.0.1:44153__je    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 869405 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [n:127.0.0.1:44153__je    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 869407 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [n:127.0.0.1:44153__je    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 869408 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [n:127.0.0.1:44153__je    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:44153__je
   [junit4]   2> 869409 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [n:127.0.0.1:44153__je    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:44153__je
   [junit4]   2> 869410 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [n:127.0.0.1:44153__je    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 869411 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [n:127.0.0.1:44153__je    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 869413 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [n:127.0.0.1:44153__je    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 869414 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [n:127.0.0.1:44153__je    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94182302675173381-127.0.0.1:44153__je-n_0000000000
   [junit4]   2> 869415 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [n:127.0.0.1:44153__je    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:44153__je
   [junit4]   2> 869415 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [n:127.0.0.1:44153__je    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 869417 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [n:127.0.0.1:44153__je    ] o.a.s.c.Overseer Overseer (id=94182302675173381-127.0.0.1:44153__je-n_0000000000) starting
   [junit4]   2> 869418 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [n:127.0.0.1:44153__je    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 869425 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [n:127.0.0.1:44153__je    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 869425 INFO  (OverseerCollectionProcessor-94182302675173381-127.0.0.1:44153__je-n_0000000000) [n:127.0.0.1:44153__je    ] o.a.s.c.OverseerCollectionProcessor Process current queue of collection creations
   [junit4]   2> 869426 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [n:127.0.0.1:44153__je    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 869429 INFO  (OverseerStateUpdate-94182302675173381-127.0.0.1:44153__je-n_0000000000) [n:127.0.0.1:44153__je    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 869430 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [n:127.0.0.1:44153__je    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 869431 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [n:127.0.0.1:44153__je    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 869432 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [n:127.0.0.1:44153__je    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_952A6D9CE6F86E0A-001/control-001/cores
   [junit4]   2> 869433 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [n:127.0.0.1:44153__je    ] o.a.s.c.SolrCore Created CoreDescriptor: {name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, loadOnStartup=true, configSetProperties=configsetprops.json, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_952A6D9CE6F86E0A-001/control-001/cores/collection1, collection=control_collection, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_952A6D9CE6F86E0A-001/control-001/cores/collection1/, coreNodeName=, dataDir=data/, shard=}
   [junit4]   2> 869433 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [n:127.0.0.1:44153__je    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_952A6D9CE6F86E0A-001/control-001/cores/collection1/
   [junit4]   2> 869433 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [n:127.0.0.1:44153__je    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 869434 INFO  (coreLoadExecutor-991-thread-1-processing-n:127.0.0.1:44153__je) [n:127.0.0.1:44153__je c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 869434 INFO  (coreLoadExecutor-991-thread-1-processing-n:127.0.0.1:44153__je) [n:127.0.0.1:44153__je c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 869435 INFO  (coreLoadExecutor-991-thread-1-processing-n:127.0.0.1:44153__je) [n:127.0.0.1:44153__je c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 869438 INFO  (OverseerStateUpdate-94182302675173381-127.0.0.1:44153__je-n_0000000000) [n:127.0.0.1:44153__je    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:44153/_je",
   [junit4]   2>   "node_name":"127.0.0.1:44153__je",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 869438 INFO  (OverseerStateUpdate-94182302675173381-127.0.0.1:44153__je-n_0000000000) [n:127.0.0.1:44153__je    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:44153/_je",
   [junit4]   2>   "node_name":"127.0.0.1:44153__je",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 869439 INFO  (OverseerStateUpdate-94182302675173381-127.0.0.1:44153__je-n_0000000000) [n:127.0.0.1:44153__je    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 869439 INFO  (OverseerStateUpdate-94182302675173381-127.0.0.1:44153__je-n_0000000000) [n:127.0.0.1:44153__je    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 869447 INFO  (zkCallback-149-thread-1-processing-n:127.0.0.1:44153__je) [n:127.0.0.1:44153__je    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 869447 INFO  (zkCallback-149-thread-2-processing-n:127.0.0.1:44153__je) [n:127.0.0.1:44153__je    ] o.a.s.c.DistributedQueue NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 869449 INFO  (zkCallback-149-thread-1-processing-n:127.0.0.1:44153__je) [n:127.0.0.1:44153__je    ] o.a.s.c.c.ZkStateReader Updated cluster state version to 1
   [junit4]   2> 869451 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [n:127.0.0.1:44153__je    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1
   [junit4]   2> 869451 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [n:127.0.0.1:44153__je    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 870435 INFO  (coreLoadExecutor-991-thread-1-processing-n:127.0.0.1:44153__je) [n:127.0.0.1:44153__je c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 870436 INFO  (coreLoadExecutor-991-thread-1-processing-n:127.0.0.1:44153__je) [n:127.0.0.1:44153__je c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 870436 INFO  (coreLoadExecutor-991-thread-1-processing-n:127.0.0.1:44153__je) [n:127.0.0.1:44153__je c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 870436 INFO  (coreLoadExecutor-991-thread-1-processing-n:127.0.0.1:44153__je) [n:127.0.0.1:44153__je c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 870437 INFO  (coreLoadExecutor-991-thread-1-processing-n:127.0.0.1:44153__je) [n:127.0.0.1:44153__je c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 870437 INFO  (coreLoadExecutor-991-thread-1-processing-n:127.0.0.1:44153__je) [n:127.0.0.1:44153__je c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_952A6D9CE6F86E0A-001/control-001/cores/collection1/'
   [junit4]   2> 870453 INFO  (coreLoadExecutor-991-thread-1-processing-n:127.0.0.1:44153__je) [n:127.0.0.1:44153__je c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 870458 INFO  (coreLoadExecutor-991-thread-1-processing-n:127.0.0.1:44153__je) [n:127.0.0.1:44153__je c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 870462 INFO  (coreLoadExecutor-991-thread-1-processing-n:127.0.0.1:44153__je) [n:127.0.0.1:44153__je c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.0.0
   [junit4]   2> 870471 INFO  (coreLoadExecutor-991-thread-1-processing-n:127.0.0.1:44153__je) [n:127.0.0.1:44153__je c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 870473 INFO  (coreLoadExecutor-991-thread-1-processing-n:127.0.0.1:44153__je) [n:127.0.0.1:44153__je c:control_collection   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 870478 INFO  (coreLoadExecutor-991-thread-1-processing-n:127.0.0.1:44153__je) [n:127.0.0.1:44153__je c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 870586 INFO  (coreLoadExecutor-991-thread-1-processing-n:127.0.0.1:44153__je) [n:127.0.0.1:44153__je c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 870594 INFO  (coreLoadExecutor-991-thread-1-processing-n:127.0.0.1:44153__je) [n:127.0.0.1:44153__je c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 870595 INFO  (coreLoadExecutor-991-thread-1-processing-n:127.0.0.1:44153__je) [n:127.0.0.1:44153__je c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 870611 INFO  (coreLoadExecutor-991-thread-1-processing-n:127.0.0.1:44153__je) [n:127.0.0.1:44153__je c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 870614 INFO  (coreLoadExecutor-991-thread-1-processing-n:127.0.0.1:44153__je) [n:127.0.0.1:44153__je c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 870616 INFO  (coreLoadExecutor-991-thread-1-processing-n:127.0.0.1:44153__je) [n:127.0.0.1:44153__je c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 870616 WARN  (coreLoadExecutor-991-thread-1-processing-n:127.0.0.1:44153__je) [n:127.0.0.1:44153__je c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 870617 WARN  (coreLoadExecutor-991-thread-1-processing-n:127.0.0.1:44153__je) [n:127.0.0.1:44153__je c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 870617 INFO  (coreLoadExecutor-991-thread-1-processing-n:127.0.0.1:44153__je) [n:127.0.0.1:44153__je c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 870617 WARN  (coreLoadExecutor-991-thread-1-processing-n:127.0.0.1:44153__je) [n:127.0.0.1:44153__je c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 870617 WARN  (coreLoadExecutor-991-thread-1-processing-n:127.0.0.1:44153__je) [n:127.0.0.1:44153__je c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 870619 INFO  (coreLoadExecutor-991-thread-1-processing-n:127.0.0.1:44153__je) [n:127.0.0.1:44153__je c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties
   [junit4]   2> org.apache.solr.core.SolrResourceNotFoundException: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1
   [junit4]   2> 	at org.apache.solr.cloud.ZkSolrResourceLoader.openResource(ZkSolrResourceLoader.java:99)
   [junit4]   2> 	at org.apache.solr.core.ConfigSetProperties.readFromResourceLoader(ConfigSetProperties.java:49)
   [junit4]   2> 	at org.apache.solr.core.ConfigSetService.createConfigSetProperties(ConfigSetService.java:114)
   [junit4]   2> 	at org.apache.solr.core.ConfigSetService.getConfig(ConfigSetService.java:76)
   [junit4]   2> 	at org.apache.solr.core.CoreContainer.create(CoreContainer.java:668)
   [junit4]   2> 	at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:397)
   [junit4]   2> 	at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:388)
   [junit4]   2> 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
   [junit4]   2> 	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:156)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 870619 INFO  (coreLoadExecutor-991-thread-1-processing-n:127.0.0.1:44153__je) [n:127.0.0.1:44153__je c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 870619 INFO  (coreLoadExecutor-991-thread-1-processing-n:127.0.0.1:44153__je) [n:127.0.0.1:44153__je c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 870620 INFO  (coreLoadExecutor-991-thread-1-processing-n:127.0.0.1:44153__je) [n:127.0.0.1:44153__je c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_952A6D9CE6F86E0A-001/control-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 870620 INFO  (coreLoadExecutor-991-thread-1-processing-n:127.0.0.1:44153__je) [n:127.0.0.1:44153__je c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@954819b
   [junit4]   2> 870620 INFO  (coreLoadExecutor-991-thread-1-processing-n:127.0.0.1:44153__je) [n:127.0.0.1:44153__je c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_952A6D9CE6F86E0A-001/control-001/cores/collection1/data
   [junit4]   2> 870620 INFO  (coreLoadExecutor-991-thread-1-processing-n:127.0.0.1:44153__je) [n:127.0.0.1:44153__je c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_952A6D9CE6F86E0A-001/control-001/cores/collection1/data/index/
   [junit4]   2> 870621 WARN  (coreLoadExecutor-991-thread-1-processing-n:127.0.0.1:44153__je) [n:127.0.0.1:44153__je c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_952A6D9CE6F86E0A-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 870621 INFO  (coreLoadExecutor-991-thread-1-processing-n:127.0.0.1:44153__je) [n:127.0.0.1:44153__je c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_952A6D9CE6F86E0A-001/control-001/cores/collection1/data/index
   [junit4]   2> 870622 INFO  (coreLoadExecutor-991-thread-1-processing-n:127.0.0.1:44153__je) [n:127.0.0.1:44153__je c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=1194497405, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
   [junit4]   2> 870622 INFO  (coreLoadExecutor-991-thread-1-processing-n:127.0.0.1:44153__je) [n:127.0.0.1:44153__je c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@294d9e8e lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@919f8e5),segFN=segments_1,generation=1}
   [junit4]   2> 870622 INFO  (coreLoadExecutor-991-thread-1-processing-n:127.0.0.1:44153__je) [n:127.0.0.1:44153__je c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 870634 INFO  (coreLoadExecutor-991-thread-1-processing-n:127.0.0.1:44153__je) [n:127.0.0.1:44153__je c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 870634 INFO  (coreLoadExecutor-991-thread-1-processing-n:127.0.0.1:44153__je) [n:127.0.0.1:44153__je c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 870635 INFO  (coreLoadExecutor-991-thread-1-processing-n:127.0.0.1:44153__je) [n:127.0.0.1:44153__je c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 870635 INFO  (coreLoadExecutor-991-thread-1-processing-n:127.0.0.1:44153__je) [n:127.0.0.1:44153__je c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 870635 INFO  (coreLoadExecutor-991-thread-1-processing-n:127.0.0.1:44153__je) [n:127.0.0.1:44153__je c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 870635 INFO  (coreLoadExecutor-991-thread-1-processing-n:127.0.0.1:44153__je) [n:127.0.0.1:44153__je c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 870635 INFO  (coreLoadExecutor-991-thread-1-processing-n:127.0.0.1:44153__je) [n:127.0.0.1:44153__je c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 870636 INFO  (coreLoadExecutor-991-thread-1-processing-n:127.0.0.1:44153__je) [n:127.0.0.1:44153__je c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 870636 INFO  (coreLoadExecutor-991-thread-1-processing-n:127.0.0.1:44153__je) [n:127.0.0.1:44153__je c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 870638 INFO  (coreLoadExecutor-991-thread-1-processing-n:127.0.0.1:44153__je) [n:127.0.0.1:44153__je c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 870639 INFO  (coreLoadExecutor-991-thread-1-processing-n:127.0.0.1:44153__je) [n:127.0.0.1:44153__je c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 870640 INFO  (coreLoadExecutor-991-thread-1-processing-n:127.0.0.1:44153__je) [n:127.0.0.1:44153__je c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 870641 INFO  (coreLoadExecutor-991-thread-1-processing-n:127.0.0.1:44153__je) [n:127.0.0.1:44153__je c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 870642 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:44153__je c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_952A6D9CE6F86E0A-001/control-001/cores/collection1/data/
   [junit4]   2> 870643 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:44153__je c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_952A6D9CE6F86E0A-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 870655 INFO  (coreLoadExecutor-991-thread-1-processing-n:127.0.0.1:44153__je) [n:127.0.0.1:44153__je c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/mbeans,standard,/update/csv,/update/json/docs,/admin/luke,/admin/segments,/get,/admin/system,/replication,/admin/properties,/config,/schema,/admin/plugins,/admin/logging,/update/json,/admin/threads,/admin/ping,/update,/admin/file
   [junit4]   2> 870657 INFO  (coreLoadExecutor-991-thread-1-processing-n:127.0.0.1:44153__je) [n:127.0.0.1:44153__je c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 870659 INFO  (coreLoadExecutor-991-thread-1-processing-n:127.0.0.1:44153__je) [n:127.0.0.1:44153__je c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 870659 INFO  (coreLoadExecutor-991-thread-1-processing-n:127.0.0.1:44153__je) [n:127.0.0.1:44153__je c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 870660 INFO  (coreLoadExecutor-991-thread-1-processing-n:127.0.0.1:44153__je) [n:127.0.0.1:44153__je c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 870660 INFO  (coreLoadExecutor-991-thread-1-processing-n:127.0.0.1:44153__je) [n:127.0.0.1:44153__je c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 870664 INFO  (coreLoadExecutor-991-thread-1-processing-n:127.0.0.1:44153__je) [n:127.0.0.1:44153__je c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=16, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.426234771183681]
   [junit4]   2> 870666 INFO  (coreLoadExecutor-991-thread-1-processing-n:127.0.0.1:44153__je) [n:127.0.0.1:44153__je c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@294d9e8e lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@919f8e5),segFN=segments_1,generation=1}
   [junit4]   2> 870666 INFO  (coreLoadExecutor-991-thread-1-processing-n:127.0.0.1:44153__je) [n:127.0.0.1:44153__je c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 870666 INFO  (coreLoadExecutor-991-thread-1-processing-n:127.0.0.1:44153__je) [n:127.0.0.1:44153__je c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@7bfd50d2[collection1] main
   [junit4]   2> 870666 INFO  (coreLoadExecutor-991-thread-1-processing-n:127.0.0.1:44153__je) [n:127.0.0.1:44153__je c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 870670 INFO  (coreLoadExecutor-991-thread-1-processing-n:127.0.0.1:44153__je) [n:127.0.0.1:44153__je c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 870670 INFO  (coreLoadExecutor-991-thread-1-processing-n:127.0.0.1:44153__je) [n:127.0.0.1:44153__je c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 870670 INFO  (coreLoadExecutor-991-thread-1-processing-n:127.0.0.1:44153__je) [n:127.0.0.1:44153__je c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 870670 INFO  (coreLoadExecutor-991-thread-1-processing-n:127.0.0.1:44153__je) [n:127.0.0.1:44153__je c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 870671 INFO  (coreLoadExecutor-991-thread-1-processing-n:127.0.0.1:44153__je) [n:127.0.0.1:44153__je c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 870671 INFO  (coreLoadExecutor-991-thread-1-processing-n:127.0.0.1:44153__je) [n:127.0.0.1:44153__je c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 870671 INFO  (coreLoadExecutor-991-thread-1-processing-n:127.0.0.1:44153__je) [n:127.0.0.1:44153__je c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 870672 INFO  (coreLoadExecutor-991-thread-1-processing-n:127.0.0.1:44153__je) [n:127.0.0.1:44153__je c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 870672 INFO  (coreLoadExecutor-991-thread-1-processing-n:127.0.0.1:44153__je) [n:127.0.0.1:44153__je c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 870673 INFO  (searcherExecutor-992-thread-1-processing-n:127.0.0.1:44153__je x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:44153__je c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@7bfd50d2[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 870673 INFO  (coreLoadExecutor-991-thread-1-processing-n:127.0.0.1:44153__je) [n:127.0.0.1:44153__je c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 870673 INFO  (coreLoadExecutor-991-thread-1-processing-n:127.0.0.1:44153__je) [n:127.0.0.1:44153__je c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 870673 INFO  (coreLoadExecutor-991-thread-1-processing-n:127.0.0.1:44153__je) [n:127.0.0.1:44153__je c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 870673 INFO  (coreLoadExecutor-991-thread-1-processing-n:127.0.0.1:44153__je) [n:127.0.0.1:44153__je c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1506916844785631232
   [junit4]   2> 870689 INFO  (coreLoadExecutor-991-thread-1-processing-n:127.0.0.1:44153__je) [n:127.0.0.1:44153__je c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 16 ms to seed version buckets with highest version 1506916844785631232
   [junit4]   2> 870690 INFO  (coreLoadExecutor-991-thread-1-processing-n:127.0.0.1:44153__je) [n:127.0.0.1:44153__je c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 870690 INFO  (coreLoadExecutor-991-thread-1-processing-n:127.0.0.1:44153__je) [n:127.0.0.1:44153__je c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 870695 INFO  (coreZkRegister-985-thread-1-processing-n:127.0.0.1:44153__je x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:44153__je c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:44153/_je collection:control_collection shard:shard1
   [junit4]   2> 870698 INFO  (coreZkRegister-985-thread-1-processing-n:127.0.0.1:44153__je x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:44153__je c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 870702 INFO  (coreZkRegister-985-thread-1-processing-n:127.0.0.1:44153__je x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:44153__je c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94182302675173381-core_node1-n_0000000000
   [junit4]   2> 870704 INFO  (coreZkRegister-985-thread-1-processing-n:127.0.0.1:44153__je x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:44153__je c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 870706 INFO  (zkCallback-149-thread-1-processing-n:127.0.0.1:44153__je) [n:127.0.0.1:44153__je    ] o.a.s.c.DistributedQueue NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 870706 INFO  (coreZkRegister-985-thread-1-processing-n:127.0.0.1:44153__je x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:44153__je c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 870707 INFO  (OverseerStateUpdate-94182302675173381-127.0.0.1:44153__je-n_0000000000) [n:127.0.0.1:44153__je    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 870707 INFO  (coreZkRegister-985-thread-1-processing-n:127.0.0.1:44153__je x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:44153__je c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 870707 INFO  (coreZkRegister-985-thread-1-processing-n:127.0.0.1:44153__je x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:44153__je c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:44153/_je/collection1/
   [junit4]   2> 870707 INFO  (coreZkRegister-985-thread-1-processing-n:127.0.0.1:44153__je x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:44153__je c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 870707 INFO  (coreZkRegister-985-thread-1-processing-n:127.0.0.1:44153__je x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:44153__je c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:44153/_je/collection1/ has no replicas
   [junit4]   2> 870708 INFO  (coreZkRegister-985-thread-1-processing-n:127.0.0.1:44153__je x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:44153__je c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:44153/_je/collection1/ shard1
   [junit4]   2> 870708 INFO  (coreZkRegister-985-thread-1-processing-n:127.0.0.1:44153__je x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:44153__je c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 870712 INFO  (zkCallback-149-thread-1-processing-n:127.0.0.1:44153__je) [n:127.0.0.1:44153__je    ] o.a.s.c.DistributedQueue NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 870713 INFO  (OverseerStateUpdate-94182302675173381-127.0.0.1:44153__je-n_0000000000) [n:127.0.0.1:44153__je    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:44153/_je",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 1
   [junit4]   2> 870816 INFO  (zkCallback-149-thread-1-processing-n:127.0.0.1:44153__je) [n:127.0.0.1:44153__je    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 870818 INFO  (zkCallback-149-thread-1-processing-n:127.0.0.1:44153__je) [n:127.0.0.1:44153__je    ] o.a.s.c.c.ZkStateReader Updated cluster state version to 2
   [junit4]   2> 870864 INFO  (coreZkRegister-985-thread-1-processing-n:127.0.0.1:44153__je x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:44153__je c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:44153/_je/collection1/ and leader is http://127.0.0.1:44153/_je/collection1/
   [junit4]   2> 870864 INFO  (coreZkRegister-985-thread-1-processing-n:127.0.0.1:44153__je x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:44153__je c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:44153/_je
   [junit4]   2> 870864 INFO  (coreZkRegister-985-thread-1-processing-n:127.0.0.1:44153__je x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:44153__je c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 870864 INFO  (coreZkRegister-985-thread-1-processing-n:127.0.0.1:44153__je x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:44153__je c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 870864 INFO  (coreZkRegister-985-thread-1-processing-n:127.0.0.1:44153__je x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:44153__je c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 870866 INFO  (zkCallback-149-thread-1-processing-n:127.0.0.1:44153__je) [n:127.0.0.1:44153__je    ] o.a.s.c.DistributedQueue NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 870867 INFO  (OverseerStateUpdate-94182302675173381-127.0.0.1:44153__je-n_0000000000) [n:127.0.0.1:44153__je    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:44153/_je",
   [junit4]   2>   "node_name":"127.0.0.1:44153__je",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"} current state version: 2
   [junit4]   2> 870868 INFO  (OverseerStateUpdate-94182302675173381-127.0.0.1:44153__je-n_0000000000) [n:127.0.0.1:44153__je    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:44153/_je",
   [junit4]   2>   "node_name":"127.0.0.1:44153__je",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 870953 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 870960 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 870971 INFO  (zkCallback-149-thread-1-processing-n:127.0.0.1:44153__je) [n:127.0.0.1:44153__je    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 870973 INFO  (zkCallback-149-thread-1-processing-n:127.0.0.1:44153__je) [n:127.0.0.1:44153__je    ] o.a.s.c.c.ZkStateReader Updated cluster state version to 3
   [junit4]   2> 870982 INFO  (zkCallback-151-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@711f439d name:ZooKeeperConnection Watcher:127.0.0.1:53435/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 870983 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 870983 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 870983 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 870986 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 871316 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_952A6D9CE6F86E0A-001/shard-1-001/cores/collection1
   [junit4]   2> 871317 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_952A6D9CE6F86E0A-001/shard-1-001
   [junit4]   2> 871318 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.e.j.s.Server jetty-9.2.11.v20150529
   [junit4]   2> 871319 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1df90dc2{/_je,null,AVAILABLE}
   [junit4]   2> 871320 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.e.j.s.ServerConnector Started ServerConnector@62a42e33{HTTP/1.1}{127.0.0.1:57477}
   [junit4]   2> 871320 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.e.j.s.Server Started @874525ms
   [junit4]   2> 871320 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_952A6D9CE6F86E0A-001/tempDir-001/jetty1, solrconfig=solrconfig.xml, hostContext=/_je, hostPort=57477, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_952A6D9CE6F86E0A-001/shard-1-001/cores}
   [junit4]   2> 871321 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@4e0e2f2a
   [junit4]   2> 871321 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_952A6D9CE6F86E0A-001/shard-1-001/'
   [junit4]   2> 871336 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_952A6D9CE6F86E0A-001/shard-1-001/solr.xml
   [junit4]   2> 871341 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_952A6D9CE6F86E0A-001/shard-1-001/cores
   [junit4]   2> 871341 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.c.CoreContainer New CoreContainer 326118734
   [junit4]   2> 871341 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_952A6D9CE6F86E0A-001/shard-1-001/]
   [junit4]   2> 871341 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_952A6D9CE6F86E0A-001/shard-1-001/lib
   [junit4]   2> 871341 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_952A6D9CE6F86E0A-001/shard-1-001/lib).
   [junit4]   2> 871349 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 871350 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 871350 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 871351 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 871351 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:53435/solr
   [junit4]   2> 871351 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 871351 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 871352 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 871361 INFO  (zkCallback-153-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4c9297ad name:ZooKeeperConnection Watcher:127.0.0.1:53435 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 871363 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 871364 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 871370 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [n:127.0.0.1:57477__je    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 871372 INFO  (zkCallback-154-thread-1-processing-n:127.0.0.1:57477__je) [n:127.0.0.1:57477__je    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@39cee76b name:ZooKeeperConnection Watcher:127.0.0.1:53435/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 871373 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [n:127.0.0.1:57477__je    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 871377 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [n:127.0.0.1:57477__je    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 872382 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [n:127.0.0.1:57477__je    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:57477__je
   [junit4]   2> 872383 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [n:127.0.0.1:57477__je    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:57477__je
   [junit4]   2> 872390 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [n:127.0.0.1:57477__je    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 872391 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [n:127.0.0.1:57477__je    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94182302675173384-127.0.0.1:57477__je-n_0000000001
   [junit4]   2> 872392 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [n:127.0.0.1:57477__je    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/94182302675173381-127.0.0.1:44153__je-n_0000000000 to know if I could be the leader
   [junit4]   2> 872393 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [n:127.0.0.1:57477__je    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 872393 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [n:127.0.0.1:57477__je    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 872394 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [n:127.0.0.1:57477__je    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_952A6D9CE6F86E0A-001/shard-1-001/cores
   [junit4]   2> 872395 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [n:127.0.0.1:57477__je    ] o.a.s.c.SolrCore Created CoreDescriptor: {name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, loadOnStartup=true, configSetProperties=configsetprops.json, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_952A6D9CE6F86E0A-001/shard-1-001/cores/collection1, collection=collection1, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_952A6D9CE6F86E0A-001/shard-1-001/cores/collection1/, coreNodeName=, dataDir=data/, shard=}
   [junit4]   2> 872395 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [n:127.0.0.1:57477__je    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_952A6D9CE6F86E0A-001/shard-1-001/cores/collection1/
   [junit4]   2> 872395 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [n:127.0.0.1:57477__je    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 872398 INFO  (coreLoadExecutor-1002-thread-1-processing-n:127.0.0.1:57477__je) [n:127.0.0.1:57477__je c:collection1   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 872398 INFO  (coreLoadExecutor-1002-thread-1-processing-n:127.0.0.1:57477__je) [n:127.0.0.1:57477__je c:collection1   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 872399 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [n:127.0.0.1:57477__je    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1
   [junit4]   2> 872399 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[952A6D9CE6F86E0A]) [n:127.0.0.1:57477__je    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 872399 INFO  (coreLoadExecutor-1002-thread-1-processing-n:127.0.0.1:57477__je) [n:127.0.0.1:57477__je c:collection1   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 872399 INFO  (zkCallback-149-thread-1-processing-n:127.0.0.1:44153__je) [n:127.0.0.1:44153__je    ] o.a.s.c.DistributedQueue NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 872400 INFO  (OverseerStateUpdate-94182302675173381-127.0.0.1:44153__je-n_0000000000) [n:127.0.0.1:44153__je    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:57477/_je",
   [junit4]   2>   "node_name":"127.0.0.1:57477__je",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "operation":"state"} current state version: 3
   [junit4]   2> 872401 INFO  (OverseerStateUpdate-94182302675173381-127.0.0.1:44153__je-n_0000000000) [n:127.0.0.1:44153__je    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:57477/_je",
   [junit4]   2>   "node_name":"127.0.0.1:57477__je",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 872401 INFO  (OverseerStateUpdate-94182302675173381-127.0.0.1:44153__je-n_0000000000) [n:127.0.0.1:44153__je    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 872401 INFO  (OverseerStateUpdate-94182302675173381-127.0.0.1:44153__je-n_0000000000) [n:127.0.0.1:44153__je    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2
   [junit4]   2> 872504 INFO  (zkCallback-151-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 872504 INFO  (zkCallback-149-thread-1-processing-n:127.0.0.1:44153__je) [n:127.0.0.1:44153__je    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 872504 INFO  (zkCallback-154-thread-1-processing-n:127.0.0.1:57477__je) [n:127.0.0.1:57477__je    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 872508 INFO  (zkCallback-151-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated cluster state version to 4
   [junit4]   2> 872508 INFO  (zkCallback-154-thread-1-processing-n:127.0.0.1:57477__je) [n:127.0.0.1:57477__je    ] o.a.s.c.c.ZkStateReader Updated cluster state version to 4
   [junit4]   2> 872508 INFO  (zkCallback-149-thread-1-processing-n:127.0.0.1:44153__je) [n:127.0.0.1:44153__je    ] o.a.s.c.c.ZkStateReader Updated cluster state version to 4
   [junit4]   2> 873400 INFO  (coreLoadExecutor-1002-thread-1-processing-n:127.0.0.1:57477__je) [n:127.0.0.1:57477__je c:collection1   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 873400 INFO  (coreLoadExecutor-1002-thread-1-processing-n:127.0.0.1:57477__je) [n:127.0.0.1:57477__je c:collection1   x:collection1] o.a.s.c.ZkController Check for collection zkNode:collection1
   [junit4]   2> 873401 INFO  (coreLoadExecutor-1002-thread-1-processing-n:127.0.0.1:57477__je) [n:127.0.0.1:57477__je c:collection1   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 873401 INFO  (coreLoadExecutor-1002-thread-1-processing-n:127.0.0.1:57477__je) [n:127.0.0.1:57477__je c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/collection1
   [junit4]   2> 873402 INFO  (coreLoadExecutor-1002-thread-1-processing-n:127.0.0.1:57477__je) [n:127.0.0.1:57477__je c:collection1   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 873402 INFO  (coreLoadExecutor-1002-thread-1-processing-n:127.0.0.1:57477__je) [n:127.0.0.1:57477__je c:collection1   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_952A6D9CE6F86E0A-001/shard-1-001/cores/collection1/'
 

[...truncated too long message...]

]   2> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> Caused by: java.lang.NullPointerException
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.blockmanagement.BlocksMap.size(BlocksMap.java:198)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.getTotalBlocks(BlockManager.java:3291)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlocksTotal(FSNamesystem.java:6223)
   [junit4]   2> 	... 53 more
   [junit4]   2> 4705059 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[952A6D9CE6F86E0A]-worker) [n:127.0.0.1:38745_ c:addReplicaColl s:shard1 r:core_node7 x:propertyDotName] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> Jul 17, 2015 1:42:06 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 2 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene53): {other_tl1=PostingsFormat(name=Asserting), range_facet_l_dv=PostingsFormat(name=LuceneVarGapDocFreqInterval), _version_=PostingsFormat(name=Asserting), multiDefault=FSTOrd50, a_t=FSTOrd50, intDefault=PostingsFormat(name=Asserting), a_i1=FSTOrd50, id=PostingsFormat(name=LuceneVarGapDocFreqInterval), range_facet_i_dv=FSTOrd50, text=Lucene50(blocksize=128), range_facet_l=FSTOrd50, timestamp=FSTOrd50}, docValues:{range_facet_l_dv=DocValuesFormat(name=Lucene50), range_facet_i_dv=DocValuesFormat(name=Memory), timestamp=DocValuesFormat(name=Memory)}, sim=RandomSimilarityProvider(queryNorm=true,coord=crazy): {}, locale=ar_KW, timezone=Etc/GMT+12
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.8.0_45 (64-bit)/cpus=4,threads=2,free=155122584,total=533725184
   [junit4]   2> NOTE: All tests run in this JVM: [SoftAutoCommitTest, RecoveryAfterSoftCommitTest, HdfsBasicDistributedZkTest, UUIDUpdateProcessorFallbackTest, CloudMLTQParserTest, PathHierarchyTokenizerFactoryTest, TestMaxScoreQueryParser, SortByFunctionTest, TestSolr4Spatial2, TestSolrQueryParser, DirectUpdateHandlerOptimizeTest, TestManagedResource, DirectSolrSpellCheckerTest, TestShardHandlerFactory, TestSurroundQueryParser, LoggingHandlerTest, TestFastOutputStream, IgnoreCommitOptimizeUpdateProcessorFactoryTest, SOLR749Test, ZkStateWriterTest, AtomicUpdatesTest, RecoveryZkTest, TestJsonRequest, SimpleFacetsTest, TestExpandComponent, TestJettySolrRunner, DistributedSuggestComponentTest, TestInitParams, TestCursorMarkWithoutUniqueKey, TestCryptoKeys, SearchHandlerTest, TestCSVResponseWriter, TestTrie, TestFieldTypeResource, TestStressRecovery, DistanceUnitsTest, TestExactSharedStatsCache, TestExtendedDismaxParser, TestPartialUpdateDeduplication, CollectionTooManyReplicasTest, SolrCoreCheckLockOnStartupTest, SampleTest, HardAutoCommitTest, SynonymTokenizerTest, TestSearcherReuse, DOMUtilTest, TestReplicationHandlerBackup, TestGroupingSearch, TestPseudoReturnFields, TestNoOpRegenerator, TestLuceneMatchVersion, DistributedQueryElevationComponentTest, OverseerTest, TestSolrJ, TestImplicitCoreProperties, BadIndexSchemaTest, TestExactStatsCache, MoreLikeThisHandlerTest, CircularListTest, OpenExchangeRatesOrgProviderTest, SpellCheckCollatorTest, TestSort, TestSchemaSimilarityResource, RemoteQueryErrorTest, RegexBoostProcessorTest, TestDefaultSimilarityFactory, AsyncMigrateRouteKeyTest, XsltUpdateRequestHandlerTest, TestSQLHandler, TestObjectReleaseTracker, ActionThrottleTest, HdfsNNFailoverTest, HdfsRecoverLeaseTest, DocValuesTest, TestPivotHelperCode, QueryEqualityTest, TestCollectionAPI, TestRebalanceLeaders, SharedFSAutoReplicaFailoverTest, SpatialRPTFieldTypeTest, TestHdfsUpdateLog, BinaryUpdateRequestHandlerTest, TestXIncludeConfig, BasicDistributedZk2Test, UnloadDistributedZkTest, LeaderElectionIntegrationTest, ShardRoutingTest, BasicZkTest, FullSolrCloudDistribCmdsTest, ClusterStateUpdateTest, TestRandomFaceting, LeaderElectionTest, ZkSolrClientTest, TestZkChroot, TestRandomDVFaceting, ShardRoutingCustomTest, AutoCommitTest, TestDistributedGrouping, TestRecovery, TestHashPartitioner, DistributedSpellCheckComponentTest, TermVectorComponentDistributedTest, TestRealTimeGet, TestStressReorder, TestReload, TestSolr4Spatial, StatsComponentTest, PeerSyncTest, ConvertedLegacyTest, TestFunctionQuery, TestLazyCores, DirectUpdateHandlerTest, OverseerCollectionProcessorTest, CurrencyFieldXmlFileTest, SolrIndexSplitterTest, SimplePostToolTest, AnalysisAfterCoreReloadTest, SignatureUpdateProcessorFactoryTest, CoreAdminHandlerTest, SolrRequestParserTest, SuggesterTest, TestStressLucene, PolyFieldTest, TestUpdate, FieldMutatingUpdateProcessorTest, TestAtomicUpdateErrorCases, DefaultValueUpdateProcessorTest, TestRemoteStreaming, DocValuesMultiTest, TestSolrDeletionPolicy1, SolrInfoMBeanTest, CacheHeaderTest, DisMaxRequestHandlerTest, TestQueryUtils, StandardRequestHandlerTest, TestReversedWildcardFilterFactory, DocumentAnalysisRequestHandlerTest, TestQueryTypes, PrimitiveFieldTypeTest, TestOmitPositions, FileBasedSpellCheckerTest, TermsComponentTest, XmlUpdateRequestHandlerTest, RequestHandlersTest, DocumentBuilderTest, TermVectorComponentTest, TestIndexingPerformance, RequiredFieldsTest, FieldAnalysisRequestHandlerTest, FastVectorHighlighterTest, IndexSchemaTest, TestJmxIntegration, TestCollationField, ReturnFieldsTest, QueryParsingTest, TestAnalyzedSuggestions, UniqFieldsUpdateProcessorFactoryTest, CSVRequestHandlerTest, TestBinaryResponseWriter, TestLFUCache, HighlighterConfigTest, TestQuerySenderListener, TestQuerySenderNoQuery, CopyFieldTest, ResponseLogComponentTest, TestMergePolicyConfig, TestDocSet, TestSearchPerf, NumericFieldsTest, TestConfig, OutputWriterTest, TestSolrCoreProperties, DirectSolrConnectionTest, NotRequiredUniqueKeyTest, SpellPossibilityIteratorTest, TestCharFilters, TestCodecSupport, TestSweetSpotSimilarityFactory, TestPerFieldSimilarity, TestLMDirichletSimilarityFactory, TestBM25SimilarityFactory, TestFastWriter, TestJmxMonitoredMap, TimeZoneUtilsTest, ResourceLoaderTest, PluginInfoTest, TestFastLRUCache, ChaosMonkeyNothingIsSafeTest, TestSystemIdResolver, PrimUtilsTest, TestSuggestSpellingConverter, DateFieldTest, RAMDirectoryFactoryTest, TestUtils, SliceStateTest, SystemInfoHandlerTest, UUIDFieldTest, FileUtilsTest, DistributedMLTComponentTest, TestRTGBase, CursorPagingTest, DistributedIntervalFacetingTest, SolrTestCaseJ4Test, TestSimpleTrackingShardHandler, AssignTest, CdcrVersionReplicationTest, CloudExitableDirectoryReaderTest, CollectionStateFormat2Test, CustomCollectionTest, DeleteReplicaTest, HttpPartitionTest, LeaderInitiatedRecoveryOnCommitTest, MultiThreadedOCPTest, TestCloudPivotFacet, TestSolrCloudWithKerberosAlt, HdfsChaosMonkeySafeLeaderTest, HdfsCollectionsAPIDistributedZkTest]
   [junit4] Completed [517/517] on J2 in 1398.18s, 1 test, 1 error <<< FAILURES!

[...truncated 12 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:533: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:474: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:61: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/extra-targets.xml:39: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build.xml:230: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/common-build.xml:513: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/common-build.xml:1426: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/common-build.xml:981: There were test failures: 517 suites, 2032 tests, 2 errors, 20 ignored (10 assumptions)

Total time: 177 minutes 54 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
No prior successful build to compare, so performing full copy of artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-NightlyTests-trunk - Build # 741 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-trunk/741/

2 tests failed.
FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=9193, name=collection1, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=9193, name=collection1, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:40813/gtjt: collection already exists: awholynewstresscollection_collection1_1
	at __randomizedtesting.SeedInfo.seed([ED2BC2B6C57EB777]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:560)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:234)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:226)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:376)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:328)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1086)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:856)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:799)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1572)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1593)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:887)


FAILED:  org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=49415, name=collection5, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=49415, name=collection5, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:51940: Could not find collection : awholynewstresscollection_collection5_1
	at __randomizedtesting.SeedInfo.seed([ED2BC2B6C57EB777]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:560)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:234)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:226)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:376)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:328)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1086)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:856)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:799)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:894)




Build Log:
[...truncated 10543 lines...]
   [junit4] Suite: org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_ED2BC2B6C57EB777-001/init-core-data-001
   [junit4]   2> 545194 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[ED2BC2B6C57EB777]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true)
   [junit4]   2> 545194 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[ED2BC2B6C57EB777]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   2> 546346 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[ED2BC2B6C57EB777]-worker) [    ] o.a.h.u.NativeCodeLoader Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
   [junit4]   1> Formatting using clusterid: testClusterID
   [junit4]   2> 547841 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[ED2BC2B6C57EB777]-worker) [    ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
   [junit4]   2> 548042 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[ED2BC2B6C57EB777]-worker) [    ] o.m.log Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
   [junit4]   2> 548053 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[ED2BC2B6C57EB777]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 548183 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[ED2BC2B6C57EB777]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 548248 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[ED2BC2B6C57EB777]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/hdfs to ./temp/Jetty_localhost_48941_hdfs____zee5uk/webapp
   [junit4]   2> 548788 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[ED2BC2B6C57EB777]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 549964 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[ED2BC2B6C57EB777]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:48941
   [junit4]   2> 551422 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[ED2BC2B6C57EB777]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 551430 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[ED2BC2B6C57EB777]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 551448 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[ED2BC2B6C57EB777]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_54026_datanode____qxxfq4/webapp
   [junit4]   2> 551705 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[ED2BC2B6C57EB777]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 552367 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[ED2BC2B6C57EB777]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:54026
   [junit4]   2> 552824 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[ED2BC2B6C57EB777]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 552826 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[ED2BC2B6C57EB777]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 552850 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[ED2BC2B6C57EB777]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_52122_datanode____h512e3/webapp
   [junit4]   2> 553035 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[ED2BC2B6C57EB777]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 554061 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[ED2BC2B6C57EB777]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:52122
   [junit4]   2> 554767 INFO  (IPC Server handler 6 on 38574) [    ] BlockStateChange BLOCK* processReport: from storage DS-d3cac151-3df0-4ebb-9d43-22e888c5c55d node DatanodeRegistration(127.0.0.1, datanodeUuid=65cb6d49-d0c8-4c27-abc7-7e9c4d8a9b3c, infoPort=52122, ipcPort=39960, storageInfo=lv=-56;cid=testClusterID;nsid=1922428434;c=0), blocks: 0, hasStaleStorages: true, processing time: 6 msecs
   [junit4]   2> 554769 INFO  (IPC Server handler 6 on 38574) [    ] BlockStateChange BLOCK* processReport: from storage DS-19856c7b-f16c-4cb6-b4ec-056cc384673f node DatanodeRegistration(127.0.0.1, datanodeUuid=65cb6d49-d0c8-4c27-abc7-7e9c4d8a9b3c, infoPort=52122, ipcPort=39960, storageInfo=lv=-56;cid=testClusterID;nsid=1922428434;c=0), blocks: 0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 554793 INFO  (IPC Server handler 4 on 38574) [    ] BlockStateChange BLOCK* processReport: from storage DS-679d4fe3-d1a1-4f94-b1d7-0657db78183b node DatanodeRegistration(127.0.0.1, datanodeUuid=0316d60a-be5d-4c66-b0ac-431c822f6527, infoPort=54026, ipcPort=40898, storageInfo=lv=-56;cid=testClusterID;nsid=1922428434;c=0), blocks: 0, hasStaleStorages: true, processing time: 10 msecs
   [junit4]   2> 554802 INFO  (IPC Server handler 4 on 38574) [    ] BlockStateChange BLOCK* processReport: from storage DS-ec700778-501f-4aeb-984c-7d575bc2efea node DatanodeRegistration(127.0.0.1, datanodeUuid=0316d60a-be5d-4c66-b0ac-431c822f6527, infoPort=54026, ipcPort=40898, storageInfo=lv=-56;cid=testClusterID;nsid=1922428434;c=0), blocks: 0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 555795 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 555796 INFO  (Thread-48101) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 555796 INFO  (Thread-48101) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 555896 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.c.ZkTestServer start zk server on port:52435
   [junit4]   2> 555897 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 555898 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 555919 INFO  (zkCallback-86-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@460f8149 name:ZooKeeperConnection Watcher:127.0.0.1:52435 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 555919 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 555920 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 555920 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 555928 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 555934 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 555954 INFO  (zkCallback-87-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@412c70e7 name:ZooKeeperConnection Watcher:127.0.0.1:52435/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 555954 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 555955 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 555955 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 555958 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 555966 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 555988 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 555996 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 556001 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 556012 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 556013 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 556014 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 556015 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 556016 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 556016 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 556021 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 556021 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 556028 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 556029 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 556030 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 556031 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 556032 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 556032 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 556034 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 556034 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 556036 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 556036 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 556037 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 556038 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 556050 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 556053 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 556061 INFO  (zkCallback-88-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@9689f63 name:ZooKeeperConnection Watcher:127.0.0.1:52435 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 556061 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 556061 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 556062 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 556066 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 556085 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 556097 INFO  (zkCallback-89-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1dd634e7 name:ZooKeeperConnection Watcher:127.0.0.1:52435/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 556097 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 556098 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 556098 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf2/solrconfig.xml
   [junit4]   2> 556099 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/solrconfig.xml
   [junit4]   2> 556117 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf2/schema.xml
   [junit4]   2> 556117 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/schema.xml
   [junit4]   2> 556119 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf2/enumsConfig.xml
   [junit4]   2> 556119 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/enumsConfig.xml
   [junit4]   2> 556121 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf2/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 556121 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 556122 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf2/stopwords.txt
   [junit4]   2> 556123 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/stopwords.txt
   [junit4]   2> 556124 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf2/protwords.txt
   [junit4]   2> 556124 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/protwords.txt
   [junit4]   2> 556126 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf2/currency.xml
   [junit4]   2> 556126 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/currency.xml
   [junit4]   2> 556127 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf2/open-exchange-rates.json
   [junit4]   2> 556128 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/open-exchange-rates.json
   [junit4]   2> 556129 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf2/mapping-ISOLatin1Accent.txt
   [junit4]   2> 556129 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/mapping-ISOLatin1Accent.txt
   [junit4]   2> 556131 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf2/old_synonyms.txt
   [junit4]   2> 556131 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/old_synonyms.txt
   [junit4]   2> 556132 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf2/synonyms.txt
   [junit4]   2> 556132 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/synonyms.txt
   [junit4]   2> 556134 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/elevate.xml to /configs/conf2/elevate.xml
   [junit4]   2> 556134 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/elevate.xml
   [junit4]   2> 556613 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_ED2BC2B6C57EB777-001/control-001/cores/collection1
   [junit4]   2> 556618 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.e.j.s.Server jetty-9.2.11.v20150529
   [junit4]   2> 556626 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@27c0036e{/,null,AVAILABLE}
   [junit4]   2> 556626 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.e.j.s.ServerConnector Started ServerConnector@2825b105{HTTP/1.1}{127.0.0.1:46359}
   [junit4]   2> 556626 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.e.j.s.Server Started @559972ms
   [junit4]   2> 556626 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=hdfs://localhost:38574/hdfs__localhost_38574__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-trunk_solr_build_solr-core_test_J0_temp_solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_ED2BC2B6C57EB777-001_tempDir-002_control_data, hostContext=/, hostPort=46359, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_ED2BC2B6C57EB777-001/control-001/cores}
   [junit4]   2> 556627 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@4e0e2f2a
   [junit4]   2> 556627 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_ED2BC2B6C57EB777-001/control-001/'
   [junit4]   2> 556665 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_ED2BC2B6C57EB777-001/control-001/solr.xml
   [junit4]   2> 556697 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_ED2BC2B6C57EB777-001/control-001/cores
   [junit4]   2> 556697 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.c.CoreContainer New CoreContainer 1300362805
   [junit4]   2> 556697 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_ED2BC2B6C57EB777-001/control-001/]
   [junit4]   2> 556698 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_ED2BC2B6C57EB777-001/control-001/lib
   [junit4]   2> 556699 WARN  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_ED2BC2B6C57EB777-001/control-001/lib).
   [junit4]   2> 556721 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 556725 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 556726 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 556727 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 556727 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:52435/solr
   [junit4]   2> 556728 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 556728 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 556729 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 556756 INFO  (zkCallback-91-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5e68829d name:ZooKeeperConnection Watcher:127.0.0.1:52435 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 556757 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 556757 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 556760 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [n:127.0.0.1:46359_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 556788 INFO  (zkCallback-92-thread-1-processing-n:127.0.0.1:46359_) [n:127.0.0.1:46359_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5f97caf0 name:ZooKeeperConnection Watcher:127.0.0.1:52435/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 556789 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [n:127.0.0.1:46359_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 556791 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [n:127.0.0.1:46359_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 556793 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [n:127.0.0.1:46359_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 556796 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [n:127.0.0.1:46359_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 556802 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [n:127.0.0.1:46359_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 556805 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [n:127.0.0.1:46359_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 556807 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [n:127.0.0.1:46359_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 556834 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [n:127.0.0.1:46359_    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 556857 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [n:127.0.0.1:46359_    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 556858 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [n:127.0.0.1:46359_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:46359_
   [junit4]   2> 556858 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [n:127.0.0.1:46359_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:46359_
   [junit4]   2> 556860 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [n:127.0.0.1:46359_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 556863 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [n:127.0.0.1:46359_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 556869 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [n:127.0.0.1:46359_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 556870 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [n:127.0.0.1:46359_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94176930417475589-127.0.0.1:46359_-n_0000000000
   [junit4]   2> 556870 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [n:127.0.0.1:46359_    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:46359_
   [junit4]   2> 556871 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [n:127.0.0.1:46359_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 556875 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [n:127.0.0.1:46359_    ] o.a.s.c.Overseer Overseer (id=94176930417475589-127.0.0.1:46359_-n_0000000000) starting
   [junit4]   2> 556877 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [n:127.0.0.1:46359_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 556885 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [n:127.0.0.1:46359_    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 556885 INFO  (OverseerCollectionProcessor-94176930417475589-127.0.0.1:46359_-n_0000000000) [n:127.0.0.1:46359_    ] o.a.s.c.OverseerCollectionProcessor Process current queue of collection creations
   [junit4]   2> 556885 INFO  (OverseerStateUpdate-94176930417475589-127.0.0.1:46359_-n_0000000000) [n:127.0.0.1:46359_    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 556886 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [n:127.0.0.1:46359_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 556896 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [n:127.0.0.1:46359_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 556897 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [n:127.0.0.1:46359_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 556898 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [n:127.0.0.1:46359_    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_ED2BC2B6C57EB777-001/control-001/cores
   [junit4]   2> 556899 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [n:127.0.0.1:46359_    ] o.a.s.c.SolrCore Created CoreDescriptor: {name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, loadOnStartup=true, configSetProperties=configsetprops.json, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_ED2BC2B6C57EB777-001/control-001/cores/collection1, collection=control_collection, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_ED2BC2B6C57EB777-001/control-001/cores/collection1/, coreNodeName=, dataDir=data/, shard=}
   [junit4]   2> 556899 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [n:127.0.0.1:46359_    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_ED2BC2B6C57EB777-001/control-001/cores/collection1/
   [junit4]   2> 556899 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [n:127.0.0.1:46359_    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 556900 INFO  (coreLoadExecutor-460-thread-1-processing-n:127.0.0.1:46359_) [n:127.0.0.1:46359_ c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 556900 INFO  (coreLoadExecutor-460-thread-1-processing-n:127.0.0.1:46359_) [n:127.0.0.1:46359_ c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 556901 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [n:127.0.0.1:46359_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0
   [junit4]   2> 556901 INFO  (coreLoadExecutor-460-thread-1-processing-n:127.0.0.1:46359_) [n:127.0.0.1:46359_ c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 556901 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [n:127.0.0.1:46359_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 556901 INFO  (OverseerStateUpdate-94176930417475589-127.0.0.1:46359_-n_0000000000) [n:127.0.0.1:46359_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:46359",
   [junit4]   2>   "node_name":"127.0.0.1:46359_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 556902 INFO  (OverseerStateUpdate-94176930417475589-127.0.0.1:46359_-n_0000000000) [n:127.0.0.1:46359_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:46359",
   [junit4]   2>   "node_name":"127.0.0.1:46359_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 556902 INFO  (OverseerStateUpdate-94176930417475589-127.0.0.1:46359_-n_0000000000) [n:127.0.0.1:46359_    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 556902 INFO  (OverseerStateUpdate-94176930417475589-127.0.0.1:46359_-n_0000000000) [n:127.0.0.1:46359_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 556903 INFO  (zkCallback-92-thread-1-processing-n:127.0.0.1:46359_) [n:127.0.0.1:46359_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 556907 INFO  (zkCallback-92-thread-1-processing-n:127.0.0.1:46359_) [n:127.0.0.1:46359_    ] o.a.s.c.c.ZkStateReader Updated cluster state version to 1
   [junit4]   2> 556908 INFO  (zkCallback-92-thread-1-processing-n:127.0.0.1:46359_) [n:127.0.0.1:46359_    ] o.a.s.c.DistributedQueue NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 557901 INFO  (coreLoadExecutor-460-thread-1-processing-n:127.0.0.1:46359_) [n:127.0.0.1:46359_ c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 557902 INFO  (coreLoadExecutor-460-thread-1-processing-n:127.0.0.1:46359_) [n:127.0.0.1:46359_ c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 557902 INFO  (coreLoadExecutor-460-thread-1-processing-n:127.0.0.1:46359_) [n:127.0.0.1:46359_ c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 557902 INFO  (coreLoadExecutor-460-thread-1-processing-n:127.0.0.1:46359_) [n:127.0.0.1:46359_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 557903 INFO  (coreLoadExecutor-460-thread-1-processing-n:127.0.0.1:46359_) [n:127.0.0.1:46359_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 557903 INFO  (coreLoadExecutor-460-thread-1-processing-n:127.0.0.1:46359_) [n:127.0.0.1:46359_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_ED2BC2B6C57EB777-001/control-001/cores/collection1/'
   [junit4]   2> 557930 INFO  (coreLoadExecutor-460-thread-1-processing-n:127.0.0.1:46359_) [n:127.0.0.1:46359_ c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 557942 INFO  (coreLoadExecutor-460-thread-1-processing-n:127.0.0.1:46359_) [n:127.0.0.1:46359_ c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 557950 INFO  (coreLoadExecutor-460-thread-1-processing-n:127.0.0.1:46359_) [n:127.0.0.1:46359_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.0.0
   [junit4]   2> 557962 INFO  (coreLoadExecutor-460-thread-1-processing-n:127.0.0.1:46359_) [n:127.0.0.1:46359_ c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 557964 INFO  (coreLoadExecutor-460-thread-1-processing-n:127.0.0.1:46359_) [n:127.0.0.1:46359_ c:control_collection   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 557973 INFO  (coreLoadExecutor-460-thread-1-processing-n:127.0.0.1:46359_) [n:127.0.0.1:46359_ c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 558161 INFO  (coreLoadExecutor-460-thread-1-processing-n:127.0.0.1:46359_) [n:127.0.0.1:46359_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 558244 INFO  (coreLoadExecutor-460-thread-1-processing-n:127.0.0.1:46359_) [n:127.0.0.1:46359_ c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 558245 INFO  (coreLoadExecutor-460-thread-1-processing-n:127.0.0.1:46359_) [n:127.0.0.1:46359_ c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 558265 INFO  (coreLoadExecutor-460-thread-1-processing-n:127.0.0.1:46359_) [n:127.0.0.1:46359_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 558268 INFO  (coreLoadExecutor-460-thread-1-processing-n:127.0.0.1:46359_) [n:127.0.0.1:46359_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 558270 INFO  (coreLoadExecutor-460-thread-1-processing-n:127.0.0.1:46359_) [n:127.0.0.1:46359_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 558271 WARN  (coreLoadExecutor-460-thread-1-processing-n:127.0.0.1:46359_) [n:127.0.0.1:46359_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 558271 WARN  (coreLoadExecutor-460-thread-1-processing-n:127.0.0.1:46359_) [n:127.0.0.1:46359_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 558271 INFO  (coreLoadExecutor-460-thread-1-processing-n:127.0.0.1:46359_) [n:127.0.0.1:46359_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 558272 WARN  (coreLoadExecutor-460-thread-1-processing-n:127.0.0.1:46359_) [n:127.0.0.1:46359_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 558272 WARN  (coreLoadExecutor-460-thread-1-processing-n:127.0.0.1:46359_) [n:127.0.0.1:46359_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 558273 INFO  (coreLoadExecutor-460-thread-1-processing-n:127.0.0.1:46359_) [n:127.0.0.1:46359_ c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties
   [junit4]   2> org.apache.solr.core.SolrResourceNotFoundException: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0
   [junit4]   2> 	at org.apache.solr.cloud.ZkSolrResourceLoader.openResource(ZkSolrResourceLoader.java:99)
   [junit4]   2> 	at org.apache.solr.core.ConfigSetProperties.readFromResourceLoader(ConfigSetProperties.java:49)
   [junit4]   2> 	at org.apache.solr.core.ConfigSetService.createConfigSetProperties(ConfigSetService.java:114)
   [junit4]   2> 	at org.apache.solr.core.ConfigSetService.getConfig(ConfigSetService.java:76)
   [junit4]   2> 	at org.apache.solr.core.CoreContainer.create(CoreContainer.java:668)
   [junit4]   2> 	at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:397)
   [junit4]   2> 	at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:388)
   [junit4]   2> 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
   [junit4]   2> 	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:156)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 558273 INFO  (coreLoadExecutor-460-thread-1-processing-n:127.0.0.1:46359_) [n:127.0.0.1:46359_ c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 558274 INFO  (coreLoadExecutor-460-thread-1-processing-n:127.0.0.1:46359_) [n:127.0.0.1:46359_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.HdfsDirectoryFactory
   [junit4]   2> 558289 INFO  (coreLoadExecutor-460-thread-1-processing-n:127.0.0.1:46359_) [n:127.0.0.1:46359_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:38574/solr_hdfs_home
   [junit4]   2> 558289 INFO  (coreLoadExecutor-460-thread-1-processing-n:127.0.0.1:46359_) [n:127.0.0.1:46359_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
   [junit4]   2> 558290 INFO  (coreLoadExecutor-460-thread-1-processing-n:127.0.0.1:46359_) [n:127.0.0.1:46359_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_ED2BC2B6C57EB777-001/control-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 558290 INFO  (coreLoadExecutor-460-thread-1-processing-n:127.0.0.1:46359_) [n:127.0.0.1:46359_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@17c93c11
   [junit4]   2> 558290 INFO  (coreLoadExecutor-460-thread-1-processing-n:127.0.0.1:46359_) [n:127.0.0.1:46359_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:38574/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 558538 INFO  (coreLoadExecutor-460-thread-1-processing-n:127.0.0.1:46359_) [n:127.0.0.1:46359_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.h.HdfsLocalityReporter Registering direcotry hdfs://localhost:38574/solr_hdfs_home/control_collection/core_node1/data for locality metrics.
   [junit4]   2> 558547 INFO  (coreLoadExecutor-460-thread-1-processing-n:127.0.0.1:46359_) [n:127.0.0.1:46359_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:38574/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 558558 INFO  (coreLoadExecutor-460-thread-1-processing-n:127.0.0.1:46359_) [n:127.0.0.1:46359_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=hdfs://localhost:38574/solr_hdfs_home/control_collection/core_node1/data/index/
   [junit4]   2> 558591 WARN  (coreLoadExecutor-460-thread-1-processing-n:127.0.0.1:46359_) [n:127.0.0.1:46359_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory 'hdfs:/localhost:38574/solr_hdfs_home/control_collection/core_node1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 558592 INFO  (coreLoadExecutor-460-thread-1-processing-n:127.0.0.1:46359_) [n:127.0.0.1:46359_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:38574/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 558618 INFO  (coreLoadExecutor-460-thread-1-processing-n:127.0.0.1:46359_) [n:127.0.0.1:46359_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.h.HdfsLocalityReporter Registering direcotry hdfs://localhost:38574/solr_hdfs_home/control_collection/core_node1/data/index for locality metrics.
   [junit4]   2> 558618 INFO  (coreLoadExecutor-460-thread-1-processing-n:127.0.0.1:46359_) [n:127.0.0.1:46359_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:38574/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 558619 INFO  (coreLoadExecutor-460-thread-1-processing-n:127.0.0.1:46359_) [n:127.0.0.1:46359_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=44, maxMergeAtOnceExplicit=17, maxMergedSegmentMB=83.142578125, floorSegmentMB=1.5390625, forceMergeDeletesPctAllowed=3.566220373364344, segmentsPerTier=17.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
   [junit4]   2> 559193 INFO  (IPC Server handler 6 on 38574) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:42746 is added to blk_1073741825_1001{blockUCState=COMMITTED, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-19856c7b-f16c-4cb6-b4ec-056cc384673f:NORMAL:127.0.0.1:54936|RBW], ReplicaUnderConstruction[[DISK]DS-679d4fe3-d1a1-4f94-b1d7-0657db78183b:NORMAL:127.0.0.1:42746|RBW]]} size 71
   [junit4]   2> 559199 INFO  (IPC Server handler 3 on 38574) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:54936 is added to blk_1073741825_1001 size 71
   [junit4]   2> 559585 INFO  (coreLoadExecutor-460-thread-1-processing-n:127.0.0.1:46359_) [n:127.0.0.1:46359_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(HdfsDirectory@b6a47b2c lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@788e69; maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 559585 INFO  (coreLoadExecutor-460-thread-1-processing-n:127.0.0.1:46359_) [n:127.0.0.1:46359_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 559592 INFO  (coreLoadExecutor-460-thread-1-processing-n:127.0.0.1:46359_) [n:127.0.0.1:46359_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 559593 INFO  (coreLoadExecutor-460-thread-1-processing-n:127.0.0.1:46359_) [n:127.0.0.1:46359_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 559593 INFO  (coreLoadExecutor-460-thread-1-processing-n:127.0.0.1:46359_) [n:127.0.0.1:46359_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 559593 INFO  (coreLoadExecutor-460-thread-1-processing-n:127.0.0.1:46359_) [n:127.0.0.1:46359_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 559593 INFO  (coreLoadExecutor-460-thread-1-processing-n:127.0.0.1:46359_) [n:127.0.0.1:46359_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 559593 INFO  (coreLoadExecutor-460-thread-1-processing-n:127.0.0.1:46359_) [n:127.0.0.1:46359_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 559593 INFO  (coreLoadExecutor-460-thread-1-processing-n:127.0.0.1:46359_) [n:127.0.0.1:46359_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 559594 INFO  (coreLoadExecutor-460-thread-1-processing-n:127.0.0.1:46359_) [n:127.0.0.1:46359_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 559594 INFO  (coreLoadExecutor-460-thread-1-processing-n:127.0.0.1:46359_) [n:127.0.0.1:46359_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 559596 INFO  (coreLoadExecutor-460-thread-1-processing-n:127.0.0.1:46359_) [n:127.0.0.1:46359_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 559597 INFO  (coreLoadExecutor-460-thread-1-processing-n:127.0.0.1:46359_) [n:127.0.0.1:46359_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 559598 INFO  (coreLoadExecutor-460-thread-1-processing-n:127.0.0.1:46359_) [n:127.0.0.1:46359_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 559599 INFO  (coreLoadExecutor-460-thread-1-processing-n:127.0.0.1:46359_) [n:127.0.0.1:46359_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 559604 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:46359_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in hdfs://localhost:38574/solr_hdfs_home/control_collection/core_node1/data/
   [junit4]   2> 559622 INFO  (coreLoadExecutor-460-thread-1-processing-n:127.0.0.1:46359_) [n:127.0.0.1:46359_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/mbeans,standard,/update/csv,/update/json/docs,/admin/luke,/admin/segments,/get,/admin/system,/replication,/admin/properties,/config,/schema,/admin/plugins,/admin/logging,/update/json,/admin/threads,/admin/ping,/update,/admin/file
   [junit4]   2> 559622 INFO  (coreLoadExecutor-460-thread-1-processing-n:127.0.0.1:46359_) [n:127.0.0.1:46359_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 559689 INFO  (coreLoadExecutor-460-thread-1-processing-n:127.0.0.1:46359_) [n:127.0.0.1:46359_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
   [junit4]   2> 559689 INFO  (coreLoadExecutor-460-thread-1-processing-n:127.0.0.1:46359_) [n:127.0.0.1:46359_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing HdfsUpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 tlogDfsReplication=2
   [junit4]   2> 559754 INFO  (coreLoadExecutor-460-thread-1-processing-n:127.0.0.1:46359_) [n:127.0.0.1:46359_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 559755 INFO  (coreLoadExecutor-460-thread-1-processing-n:127.0.0.1:46359_) [n:127.0.0.1:46359_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 559763 INFO  (coreLoadExecutor-460-thread-1-processing-n:127.0.0.1:46359_) [n:127.0.0.1:46359_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=1742926484, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
   [junit4]   2> 559917 INFO  (coreLoadExecutor-460-thread-1-processing-n:127.0.0.1:46359_) [n:127.0.0.1:46359_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(HdfsDirectory@b6a47b2c lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@788e69; maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 559918 INFO  (coreLoadExecutor-460-thread-1-processing-n:127.0.0.1:46359_) [n:127.0.0.1:46359_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 559918 INFO  (coreLoadExecutor-460-thread-1-processing-n:127.0.0.1:46359_) [n:127.0.0.1:46359_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@154d5969[collection1] main
   [junit4]   2> 559932 INFO  (coreLoadExecutor-460-thread-1-processing-n:127.0.0.1:46359_) [n:127.0.0.1:46359_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 559933 INFO  (coreLoadExecutor-460-thread-1-processing-n:127.0.0.1:46359_) [n:127.0.0.1:46359_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 559933 INFO  (coreLoadExecutor-460-thread-1-processing-n:127.0.0.1:46359_) [n:127.0.0.1:46359_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 559933 INFO  (coreLoadExecutor-460-thread-1-processing-n:127.0.0.1:46359_) [n:127.0.0.1:46359_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 559934 INFO  (coreLoadExecutor-460-thread-1-processing-n:127.0.0.1:46359_) [n:127.0.0.1:46359_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 559934 INFO  (coreLoadExecutor-460-thread-1-processing-n:127.0.0.1:46359_) [n:127.0.0.1:46359_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 559934 INFO  (coreLoadExecutor-460-thread-1-processing-n:127.0.0.1:46359_) [n:127.0.0.1:46359_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 559935 INFO  (coreLoadExecutor-460-thread-1-processing-n:127.0.0.1:46359_) [n:127.0.0.1:46359_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 559935 INFO  (coreLoadExecutor-460-thread-1-processing-n:127.0.0.1:46359_) [n:127.0.0.1:46359_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 559935 INFO  (coreLoadExecutor-460-thread-1-processing-n:127.0.0.1:46359_) [n:127.0.0.1:46359_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 559942 INFO  (searcherExecutor-461-thread-1-processing-n:127.0.0.1:46359_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:46359_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@154d5969[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 559955 INFO  (coreLoadExecutor-460-thread-1-processing-n:127.0.0.1:46359_) [n:127.0.0.1:46359_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 559955 INFO  (coreLoadExecutor-460-thread-1-processing-n:127.0.0.1:46359_) [n:127.0.0.1:46359_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 256 version buckets from index
   [junit4]   2> 559955 INFO  (coreLoadExecutor-460-thread-1-processing-n:127.0.0.1:46359_) [n:127.0.0.1:46359_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 559956 INFO  (coreLoadExecutor-460-thread-1-processing-n:127.0.0.1:46359_) [n:127.0.0.1:46359_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1506830891040636928
   [junit4]   2> 559956 INFO  (coreLoadExecutor-460-thread-1-processing-n:127.0.0.1:46359_) [n:127.0.0.1:46359_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 1 ms to seed version buckets with highest version 1506830891040636928
   [junit4]   2> 559956 INFO  (coreLoadExecutor-460-thread-1-processing-n:127.0.0.1:46359_) [n:127.0.0.1:46359_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 559957 INFO  (coreLoadExecutor-460-thread-1-processing-n:127.0.0.1:46359_) [n:127.0.0.1:46359_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 559957 INFO  (coreZkRegister-454-thread-1-processing-n:127.0.0.1:46359_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:46359_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:46359 collection:control_collection shard:shard1
   [junit4]   2> 559962 INFO  (coreZkRegister-454-thread-1-processing-n:127.0.0.1:46359_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:46359_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 559966 INFO  (coreZkRegister-454-thread-1-processing-n:127.0.0.1:46359_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:46359_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94176930417475589-core_node1-n_0000000000
   [junit4]   2> 559968 INFO  (coreZkRegister-454-thread-1-processing-n:127.0.0.1:46359_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:46359_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 559990 INFO  (zkCallback-92-thread-1-processing-n:127.0.0.1:46359_) [n:127.0.0.1:46359_    ] o.a.s.c.DistributedQueue NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 559991 INFO  (OverseerStateUpdate-94176930417475589-127.0.0.1:46359_-n_0000000000) [n:127.0.0.1:46359_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 559992 INFO  (zkCallback-92-thread-1-processing-n:127.0.0.1:46359_) [n:127.0.0.1:46359_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 559995 INFO  (zkCallback-92-thread-1-processing-n:127.0.0.1:46359_) [n:127.0.0.1:46359_    ] o.a.s.c.c.ZkStateReader Updated cluster state version to 2
   [junit4]   2> 559997 INFO  (coreZkRegister-454-thread-1-processing-n:127.0.0.1:46359_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:46359_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 560000 INFO  (coreZkRegister-454-thread-1-processing-n:127.0.0.1:46359_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:46359_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 560000 INFO  (coreZkRegister-454-thread-1-processing-n:127.0.0.1:46359_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:46359_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:46359/collection1/
   [junit4]   2> 560001 INFO  (coreZkRegister-454-thread-1-processing-n:127.0.0.1:46359_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:46359_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 560001 INFO  (coreZkRegister-454-thread-1-processing-n:127.0.0.1:46359_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:46359_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:46359/collection1/ has no replicas
   [junit4]   2> 560001 INFO  (coreZkRegister-454-thread-1-processing-n:127.0.0.1:46359_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:46359_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:46359/collection1/ shard1
   [junit4]   2> 560001 INFO  (coreZkRegister-454-thread-1-processing-n:127.0.0.1:46359_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:46359_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 560005 INFO  (zkCallback-92-thread-1-processing-n:127.0.0.1:46359_) [n:127.0.0.1:46359_    ] o.a.s.c.DistributedQueue NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 560006 INFO  (OverseerStateUpdate-94176930417475589-127.0.0.1:46359_-n_0000000000) [n:127.0.0.1:46359_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:46359",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 2
   [junit4]   2> 560109 INFO  (zkCallback-92-thread-1-processing-n:127.0.0.1:46359_) [n:127.0.0.1:46359_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 560112 INFO  (zkCallback-92-thread-1-processing-n:127.0.0.1:46359_) [n:127.0.0.1:46359_    ] o.a.s.c.c.ZkStateReader Updated cluster state version to 3
   [junit4]   2> 560161 INFO  (coreZkRegister-454-thread-1-processing-n:127.0.0.1:46359_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:46359_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:46359/collection1/ and leader is http://127.0.0.1:46359/collection1/
   [junit4]   2> 560161 INFO  (coreZkRegister-454-thread-1-processing-n:127.0.0.1:46359_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:46359_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:46359
   [junit4]   2> 560161 INFO  (coreZkRegister-454-thread-1-processing-n:127.0.0.1:46359_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:46359_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 560162 INFO  (coreZkRegister-454-thread-1-processing-n:127.0.0.1:46359_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:46359_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 560162 INFO  (coreZkRegister-454-thread-1-processing-n:127.0.0.1:46359_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:46359_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 560163 INFO  (zkCallback-92-thread-1-processing-n:127.0.0.1:46359_) [n:127.0.0.1:46359_    ] o.a.s.c.DistributedQueue NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 560164 INFO  (OverseerStateUpdate-94176930417475589-127.0.0.1:46359_-n_0000000000) [n:127.0.0.1:46359_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:46359",
   [junit4]   2>   "node_name":"127.0.0.1:46359_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"} current state version: 3
   [junit4]   2> 560165 INFO  (OverseerStateUpdate-94176930417475589-127.0.0.1:46359_-n_0000000000) [n:127.0.0.1:46359_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:46359",
   [junit4]   2>   "node_name":"127.0.0.1:46359_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 560274 INFO  (zkCallback-92-thread-1-processing-n:127.0.0.1:46359_) [n:127.0.0.1:46359_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 560281 INFO  (zkCallback-92-thread-1-processing-n:127.0.0.1:46359_) [n:127.0.0.1:46359_    ] o.a.s.c.c.ZkStateReader Updated cluster state version to 4
   [junit4]   2> 560406 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 560408 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 560421 INFO  (zkCallback-94-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5cb9cd92 name:ZooKeeperConnection Watcher:127.0.0.1:52435/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 560421 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 560422 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 560422 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 560441 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 560442 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with stateFormat=2
   [junit4]   2> 560442 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 560442 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 560446 INFO  (zkCallback-95-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@32065045 name:ZooKeeperConnection Watcher:127.0.0.1:52435/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 560446 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 560446 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 560449 INFO  (zkCallback-92-thread-1-processing-n:127.0.0.1:46359_) [n:127.0.0.1:46359_    ] o.a.s.c.DistributedQueue NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 560454 INFO  (OverseerStateUpdate-94176930417475589-127.0.0.1:46359_-n_0000000000) [n:127.0.0.1:46359_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"create",
   [junit4]   2>   "name":"collection1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "stateFormat":"2"} current state version: 4
   [junit4]   2> 560455 INFO  (OverseerStateUpdate-94176930417475589-127.0.0.1:46359_-n_0000000000) [n:127.0.0.1:46359_    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 560581 INFO  (OverseerStateUpdate-94176930417475589-127.0.0.1:46359_-n_0000000000) [n:127.0.0.1:46359_    ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/collection1/state.json
   [junit4]   2> 560589 INFO  (zkCallback-92-thread-1-processing-n:127.0.0.1:46359_) [n:127.0.0.1:46359_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 560589 INFO  (zkCallback-94-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 560592 INFO  (zkCallback-94-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated cluster state version to 5
   [junit4]   2> 560593 INFO  (zkCallback-92-thread-1-processing-n:127.0.0.1:46359_) [n:127.0.0.1:46359_    ] o.a.s.c.c.ZkStateReader Updated cluster state version to 5
   [junit4]   2> 561034 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_ED2BC2B6C57EB777-001/shard-1-001/cores/collection1
   [junit4]   2> 561035 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_ED2BC2B6C57EB777-001/shard-1-001
   [junit4]   2> 561036 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.e.j.s.Server jetty-9.2.11.v20150529
   [junit4]   2> 561039 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@730fc2dc{/,null,AVAILABLE}
   [junit4]   2> 561039 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.e.j.s.ServerConnector Started ServerConnector@78dd55c8{HTTP/1.1}{127.0.0.1:51940}
   [junit4]   2> 561040 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.e.j.s.Server Started @564385ms
   [junit4]   2> 561040 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=hdfs://localhost:38574/hdfs__localhost_38574__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-trunk_solr_build_solr-core_test_J0_temp_solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_ED2BC2B6C57EB777-001_tempDir-002_jetty1, solrconfig=solrconfig.xml, hostContext=/, hostPort=51940, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_ED2BC2B6C57EB777-001/shard-1-001/cores}
   [junit4]   2> 561040 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@4e0e2f2a
   [junit4]   2> 561041 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_ED2BC2B6C57EB777-001/shard-1-001/'
   [junit4]   2> 561080 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_ED2BC2B6C57EB777-001/shard-1-001/solr.xml
   [junit4]   2> 561101 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_ED2BC2B6C57EB777-001/shard-1-001/cores
   [junit4]   2> 561102 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.c.CoreContainer New CoreContainer 1150080825
   [junit4]   2> 561102 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_ED2BC2B6C57EB777-001/shard-1-001/]
   [junit4]   2> 561102 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_ED2BC2B6C57EB777-001/shard-1-001/lib
   [junit4]   2> 561102 WARN  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_ED2BC2B6C57EB777-001/shard-1-001/lib).
   [junit4]   2> 561114 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 561119 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 561120 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 561120 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 561121 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:52435/solr
   [junit4]   2> 561121 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 561121 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 561121 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[ED2BC2B6C57EB777]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 561131 INFO  (zkCallback-97-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7d1fc40d

[...truncated too long message...]

rj.SolrClient.request(SolrClient.java:1220)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1572)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1593)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:887)Throwable #4: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=9197, name=collection5, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    > Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:55554/gtjt: Could not find collection : awholynewstresscollection_collection5_1
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([ED2BC2B6C57EB777]:0)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:560)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:234)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:226)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:376)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:328)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1086)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:856)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:799)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:894)Throwable #5: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=9194, name=collection2, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    > Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:55554/gtjt: Could not find collection : awholynewstresscollection_collection2_1
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([ED2BC2B6C57EB777]:0)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:560)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:234)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:226)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:376)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:328)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1086)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:856)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:799)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:894)Throwable #6: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=9192, name=collection0, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    > Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:55554/gtjt: Could not find collection : awholynewstresscollection_collection0_1
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([ED2BC2B6C57EB777]:0)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:560)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:234)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:226)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:376)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:328)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1086)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:856)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:799)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:894)
   [junit4]   2> 1747589 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[ED2BC2B6C57EB777]-worker) [n:127.0.0.1:55554_gtjt c:addReplicaColl s:shard2 r:core_node3 x:addReplicaColl_shard2_replica2] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene53): {other_tl1=FST50, range_facet_l_dv=PostingsFormat(name=Asserting), _version_=FST50, multiDefault=PostingsFormat(name=LuceneVarGapFixedInterval), a_t=PostingsFormat(name=LuceneVarGapFixedInterval), intDefault=FST50, a_i1=PostingsFormat(name=LuceneVarGapFixedInterval), id=PostingsFormat(name=Asserting), range_facet_i_dv=PostingsFormat(name=LuceneVarGapFixedInterval), text=Lucene50(blocksize=128), range_facet_l=PostingsFormat(name=LuceneVarGapFixedInterval), timestamp=PostingsFormat(name=LuceneVarGapFixedInterval)}, docValues:{range_facet_l_dv=DocValuesFormat(name=Memory), range_facet_i_dv=DocValuesFormat(name=Lucene50), timestamp=DocValuesFormat(name=Lucene50)}, sim=RandomSimilarityProvider(queryNorm=false,coord=no): {}, locale=no, timezone=Africa/Malabo
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.8.0_45 (64-bit)/cpus=4,threads=1,free=163518824,total=530579456
   [junit4]   2> NOTE: All tests run in this JVM: [UUIDUpdateProcessorFallbackTest, BinaryUpdateRequestHandlerTest, TestTrie, AnalyticsMergeStrategyTest, TestSchemaSimilarityResource, TestMaxScoreQueryParser, SuggesterTest, AutoCommitTest, TestPerFieldSimilarity, NumericFieldsTest, DOMUtilTest, TestCursorMarkWithoutUniqueKey, DirectUpdateHandlerOptimizeTest, DocExpirationUpdateProcessorFactoryTest, TestJsonRequest, QueryEqualityTest, DistributedQueryElevationComponentTest, CdcrVersionReplicationTest, BasicDistributedZk2Test, TestDocSet, TestClassNameShortening, AtomicUpdatesTest, TestCSVResponseWriter, RulesTest, RecoveryZkTest, TestImplicitCoreProperties, TestRebalanceLeaders, LeaderInitiatedRecoveryOnCommitTest, TestJettySolrRunner, PluginInfoTest, TestCryptoKeys, TestDynamicLoading, TestPseudoReturnFields, TestOverriddenPrefixQueryForCustomFieldType, SolrCoreCheckLockOnStartupTest, MultiThreadedOCPTest, TestMacros, ChangedSchemaMergeTest, TestStressReorder, TestUniqueKeyFieldResource, TestReqParamsAPI, SaslZkACLProviderTest, TestBinaryResponseWriter, PrimUtilsTest, FileBasedSpellCheckerTest, BlockDirectoryTest, TestCopyFieldCollectionResource, TestSearcherReuse, TestRequestStatusCollectionAPI, TestMissingGroups, TestFieldTypeResource, LoggingHandlerTest, SharedFSAutoReplicaFailoverTest, OverseerCollectionProcessorTest, TestZkChroot, OutputWriterTest, HardAutoCommitTest, OverseerTest, CoreMergeIndexesAdminHandlerTest, HdfsRecoverLeaseTest, SynonymTokenizerTest, TestSurroundQueryParser, TestLuceneMatchVersion, TestSQLHandler, RankQueryTest, TestSort, TestSolrQueryParserDefaultOperatorResource, TestCloudPivotFacet, TestCollationField, TestSolrJ, SpatialRPTFieldTypeTest, TestRawResponseWriter, TestInitParams, TestReload, LeaderElectionTest, TestSolrCloudWithKerberosAlt, CollectionTooManyReplicasTest, RemoteQueryErrorTest, ActionThrottleTest, TestShortCircuitedRequests, UnloadDistributedZkTest, CollectionsAPIDistributedZkTest]
   [junit4] Completed [187/517] on J2 in 340.51s, 1 test, 1 error <<< FAILURES!

[...truncated 1055 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:533: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:474: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:61: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/extra-targets.xml:39: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build.xml:230: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/common-build.xml:513: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/common-build.xml:1426: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/common-build.xml:981: There were test failures: 517 suites, 2032 tests, 2 errors, 20 ignored (10 assumptions)

Total time: 190 minutes 13 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
No prior successful build to compare, so performing full copy of artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-NightlyTests-trunk - Build # 740 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-trunk/740/

3 tests failed.
FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=2712, name=collection4, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=2712, name=collection4, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:49901/jbgv/oy: collection already exists: awholynewstresscollection_collection4_0
	at __randomizedtesting.SeedInfo.seed([A108012B41E9020D]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:560)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:234)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:226)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:376)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:328)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1086)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:856)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:799)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1572)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1593)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:887)


FAILED:  org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=64964, name=collection4, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=64964, name=collection4, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:41345/zqga/v: Could not find collection : awholynewstresscollection_collection4_0
	at __randomizedtesting.SeedInfo.seed([A108012B41E9020D]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:560)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:234)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:226)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:376)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:328)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1086)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:856)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:799)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:894)


FAILED:  org.apache.solr.handler.TestReplicationHandler.doTestReplicateAfterCoreReload

Error Message:
expected:<[{indexVersion=1436939928071,generation=2,filelist=[_0.fdt, _0.fdx, _0.fnm, _0.nvd, _0.nvm, _0.si, _0_BlockTreeOrds_0.doc, _0_BlockTreeOrds_0.tio, _0_BlockTreeOrds_0.tipo, _4.fdt, _4.fdx, _4.fnm, _4.nvd, _4.nvm, _4.si, _4_BlockTreeOrds_0.doc, _4_BlockTreeOrds_0.tio, _4_BlockTreeOrds_0.tipo, _6.fdt, _6.fdx, _6.fnm, _6.nvd, _6.nvm, _6.si, _6_BlockTreeOrds_0.doc, _6_BlockTreeOrds_0.tio, _6_BlockTreeOrds_0.tipo, _7.fdt, _7.fdx, _7.fnm, _7.nvd, _7.nvm, _7.si, _7_BlockTreeOrds_0.doc, _7_BlockTreeOrds_0.tio, _7_BlockTreeOrds_0.tipo, segments_2]}]> but was:<[{indexVersion=1436939928071,generation=2,filelist=[_0.fdt, _0.fdx, _0.fnm, _0.nvd, _0.nvm, _0.si, _0_BlockTreeOrds_0.doc, _0_BlockTreeOrds_0.tio, _0_BlockTreeOrds_0.tipo, _4.fdt, _4.fdx, _4.fnm, _4.nvd, _4.nvm, _4.si, _4_BlockTreeOrds_0.doc, _4_BlockTreeOrds_0.tio, _4_BlockTreeOrds_0.tipo, _6.fdt, _6.fdx, _6.fnm, _6.nvd, _6.nvm, _6.si, _6_BlockTreeOrds_0.doc, _6_BlockTreeOrds_0.tio, _6_BlockTreeOrds_0.tipo, _7.fdt, _7.fdx, _7.fnm, _7.nvd, _7.nvm, _7.si, _7_BlockTreeOrds_0.doc, _7_BlockTreeOrds_0.tio, _7_BlockTreeOrds_0.tipo, segments_2]}, {indexVersion=1436939928071,generation=3,filelist=[_0.fdt, _0.fdx, _0.fnm, _0.nvd, _0.nvm, _0.si, _0_BlockTreeOrds_0.doc, _0_BlockTreeOrds_0.tio, _0_BlockTreeOrds_0.tipo, _8.cfe, _8.cfs, _8.si, segments_3]}]>

Stack Trace:
java.lang.AssertionError: expected:<[{indexVersion=1436939928071,generation=2,filelist=[_0.fdt, _0.fdx, _0.fnm, _0.nvd, _0.nvm, _0.si, _0_BlockTreeOrds_0.doc, _0_BlockTreeOrds_0.tio, _0_BlockTreeOrds_0.tipo, _4.fdt, _4.fdx, _4.fnm, _4.nvd, _4.nvm, _4.si, _4_BlockTreeOrds_0.doc, _4_BlockTreeOrds_0.tio, _4_BlockTreeOrds_0.tipo, _6.fdt, _6.fdx, _6.fnm, _6.nvd, _6.nvm, _6.si, _6_BlockTreeOrds_0.doc, _6_BlockTreeOrds_0.tio, _6_BlockTreeOrds_0.tipo, _7.fdt, _7.fdx, _7.fnm, _7.nvd, _7.nvm, _7.si, _7_BlockTreeOrds_0.doc, _7_BlockTreeOrds_0.tio, _7_BlockTreeOrds_0.tipo, segments_2]}]> but was:<[{indexVersion=1436939928071,generation=2,filelist=[_0.fdt, _0.fdx, _0.fnm, _0.nvd, _0.nvm, _0.si, _0_BlockTreeOrds_0.doc, _0_BlockTreeOrds_0.tio, _0_BlockTreeOrds_0.tipo, _4.fdt, _4.fdx, _4.fnm, _4.nvd, _4.nvm, _4.si, _4_BlockTreeOrds_0.doc, _4_BlockTreeOrds_0.tio, _4_BlockTreeOrds_0.tipo, _6.fdt, _6.fdx, _6.fnm, _6.nvd, _6.nvm, _6.si, _6_BlockTreeOrds_0.doc, _6_BlockTreeOrds_0.tio, _6_BlockTreeOrds_0.tipo, _7.fdt, _7.fdx, _7.fnm, _7.nvd, _7.nvm, _7.si, _7_BlockTreeOrds_0.doc, _7_BlockTreeOrds_0.tio, _7_BlockTreeOrds_0.tipo, segments_2]}, {indexVersion=1436939928071,generation=3,filelist=[_0.fdt, _0.fdx, _0.fnm, _0.nvd, _0.nvm, _0.si, _0_BlockTreeOrds_0.doc, _0_BlockTreeOrds_0.tio, _0_BlockTreeOrds_0.tipo, _8.cfe, _8.cfs, _8.si, segments_3]}]>
	at __randomizedtesting.SeedInfo.seed([A108012B41E9020D:84DF1A1B31A10C0E]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.failNotEquals(Assert.java:647)
	at org.junit.Assert.assertEquals(Assert.java:128)
	at org.junit.Assert.assertEquals(Assert.java:147)
	at org.apache.solr.handler.TestReplicationHandler.doTestReplicateAfterCoreReload(TestReplicationHandler.java:1138)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:483)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 10313 lines...]
   [junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A108012B41E9020D-001/init-core-data-001
   [junit4]   2> 282976 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[A108012B41E9020D]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (true)
   [junit4]   2> 282976 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[A108012B41E9020D]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /jbgv/oy
   [junit4]   2> 282987 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 282990 INFO  (Thread-1574) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 282991 INFO  (Thread-1574) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 283090 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [    ] o.a.s.c.ZkTestServer start zk server on port:58829
   [junit4]   2> 283091 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 283091 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 283098 INFO  (zkCallback-151-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@234ba257 name:ZooKeeperConnection Watcher:127.0.0.1:58829 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 283099 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 283099 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 283099 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 283110 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 283111 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 283123 INFO  (zkCallback-152-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@53081ded name:ZooKeeperConnection Watcher:127.0.0.1:58829/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 283123 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 283123 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 283123 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 283125 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 283126 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 283127 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 283128 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 283128 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 283130 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 283131 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 283132 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 283132 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 283133 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 283134 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 283135 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 283135 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 283136 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 283137 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 283138 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 283138 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 283139 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 283139 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 283141 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 283141 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 283142 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 283142 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 283144 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 283144 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 283146 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 283146 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 283148 INFO  (zkCallback-153-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@672b505a name:ZooKeeperConnection Watcher:127.0.0.1:58829/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 283148 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 283148 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 283459 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A108012B41E9020D-001/control-001/cores/collection1
   [junit4]   2> 283461 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [    ] o.e.j.s.Server jetty-9.2.11.v20150529
   [junit4]   2> 283462 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@188ac5db{/jbgv/oy,null,AVAILABLE}
   [junit4]   2> 283496 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [    ] o.e.j.s.ServerConnector Started ServerConnector@3aab80c1{SSL-http/1.1}{127.0.0.1:38656}
   [junit4]   2> 283496 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [    ] o.e.j.s.Server Started @287107ms
   [junit4]   2> 283496 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A108012B41E9020D-001/tempDir-001/control/data, hostContext=/jbgv/oy, hostPort=38656, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A108012B41E9020D-001/control-001/cores}
   [junit4]   2> 283498 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@c387f44
   [junit4]   2> 283498 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A108012B41E9020D-001/control-001/'
   [junit4]   2> 283520 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A108012B41E9020D-001/control-001/solr.xml
   [junit4]   2> 283525 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A108012B41E9020D-001/control-001/cores
   [junit4]   2> 283525 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [    ] o.a.s.c.CoreContainer New CoreContainer 863378522
   [junit4]   2> 283525 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A108012B41E9020D-001/control-001/]
   [junit4]   2> 283525 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A108012B41E9020D-001/control-001/lib
   [junit4]   2> 283526 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A108012B41E9020D-001/control-001/lib).
   [junit4]   2> 283534 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 283537 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 283538 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 283539 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 283539 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:58829/solr
   [junit4]   2> 283539 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 283539 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 283540 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 283545 INFO  (zkCallback-155-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@655b8611 name:ZooKeeperConnection Watcher:127.0.0.1:58829 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 283546 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 283546 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 283549 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [n:127.0.0.1:38656_jbgv%2Foy    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 283550 INFO  (zkCallback-156-thread-1-processing-n:127.0.0.1:38656_jbgv%2Foy) [n:127.0.0.1:38656_jbgv%2Foy    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@67701666 name:ZooKeeperConnection Watcher:127.0.0.1:58829/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 283551 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [n:127.0.0.1:38656_jbgv%2Foy    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 283552 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [n:127.0.0.1:38656_jbgv%2Foy    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 283554 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [n:127.0.0.1:38656_jbgv%2Foy    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 283555 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [n:127.0.0.1:38656_jbgv%2Foy    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 283557 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [n:127.0.0.1:38656_jbgv%2Foy    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 283559 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [n:127.0.0.1:38656_jbgv%2Foy    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 283561 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [n:127.0.0.1:38656_jbgv%2Foy    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 283566 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [n:127.0.0.1:38656_jbgv%2Foy    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 283567 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [n:127.0.0.1:38656_jbgv%2Foy    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 283568 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [n:127.0.0.1:38656_jbgv%2Foy    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:38656_jbgv%2Foy
   [junit4]   2> 283569 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [n:127.0.0.1:38656_jbgv%2Foy    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:38656_jbgv%2Foy
   [junit4]   2> 283570 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [n:127.0.0.1:38656_jbgv%2Foy    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 283571 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [n:127.0.0.1:38656_jbgv%2Foy    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 283572 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [n:127.0.0.1:38656_jbgv%2Foy    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 283573 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [n:127.0.0.1:38656_jbgv%2Foy    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94171179046076420-127.0.0.1:38656_jbgv%2Foy-n_0000000000
   [junit4]   2> 283573 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [n:127.0.0.1:38656_jbgv%2Foy    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:38656_jbgv%2Foy
   [junit4]   2> 283573 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [n:127.0.0.1:38656_jbgv%2Foy    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 283575 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [n:127.0.0.1:38656_jbgv%2Foy    ] o.a.s.c.Overseer Overseer (id=94171179046076420-127.0.0.1:38656_jbgv%2Foy-n_0000000000) starting
   [junit4]   2> 283576 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [n:127.0.0.1:38656_jbgv%2Foy    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 283581 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [n:127.0.0.1:38656_jbgv%2Foy    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 283582 INFO  (OverseerStateUpdate-94171179046076420-127.0.0.1:38656_jbgv%2Foy-n_0000000000) [n:127.0.0.1:38656_jbgv%2Foy    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 283583 INFO  (OverseerCollectionProcessor-94171179046076420-127.0.0.1:38656_jbgv%2Foy-n_0000000000) [n:127.0.0.1:38656_jbgv%2Foy    ] o.a.s.c.OverseerCollectionProcessor Process current queue of collection creations
   [junit4]   2> 283584 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [n:127.0.0.1:38656_jbgv%2Foy    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 283586 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [n:127.0.0.1:38656_jbgv%2Foy    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 283587 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [n:127.0.0.1:38656_jbgv%2Foy    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 283587 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [n:127.0.0.1:38656_jbgv%2Foy    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A108012B41E9020D-001/control-001/cores
   [junit4]   2> 283588 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [n:127.0.0.1:38656_jbgv%2Foy    ] o.a.s.c.SolrCore Created CoreDescriptor: {name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, loadOnStartup=true, configSetProperties=configsetprops.json, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A108012B41E9020D-001/control-001/cores/collection1, collection=control_collection, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A108012B41E9020D-001/control-001/cores/collection1/, coreNodeName=, dataDir=data/, shard=}
   [junit4]   2> 283589 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [n:127.0.0.1:38656_jbgv%2Foy    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A108012B41E9020D-001/control-001/cores/collection1/
   [junit4]   2> 283589 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [n:127.0.0.1:38656_jbgv%2Foy    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 283590 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:38656_jbgv%2Foy) [n:127.0.0.1:38656_jbgv%2Foy c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 283590 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:38656_jbgv%2Foy) [n:127.0.0.1:38656_jbgv%2Foy c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 283590 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:38656_jbgv%2Foy) [n:127.0.0.1:38656_jbgv%2Foy c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 283594 INFO  (zkCallback-156-thread-1-processing-n:127.0.0.1:38656_jbgv%2Foy) [n:127.0.0.1:38656_jbgv%2Foy    ] o.a.s.c.DistributedQueue NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 283595 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [n:127.0.0.1:38656_jbgv%2Foy    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1
   [junit4]   2> 283595 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [n:127.0.0.1:38656_jbgv%2Foy    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 283595 INFO  (OverseerStateUpdate-94171179046076420-127.0.0.1:38656_jbgv%2Foy-n_0000000000) [n:127.0.0.1:38656_jbgv%2Foy    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:38656/jbgv/oy",
   [junit4]   2>   "node_name":"127.0.0.1:38656_jbgv%2Foy",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 283596 INFO  (OverseerStateUpdate-94171179046076420-127.0.0.1:38656_jbgv%2Foy-n_0000000000) [n:127.0.0.1:38656_jbgv%2Foy    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:38656/jbgv/oy",
   [junit4]   2>   "node_name":"127.0.0.1:38656_jbgv%2Foy",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 283596 INFO  (OverseerStateUpdate-94171179046076420-127.0.0.1:38656_jbgv%2Foy-n_0000000000) [n:127.0.0.1:38656_jbgv%2Foy    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 283597 INFO  (OverseerStateUpdate-94171179046076420-127.0.0.1:38656_jbgv%2Foy-n_0000000000) [n:127.0.0.1:38656_jbgv%2Foy    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 283598 INFO  (zkCallback-156-thread-1-processing-n:127.0.0.1:38656_jbgv%2Foy) [n:127.0.0.1:38656_jbgv%2Foy    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 283599 INFO  (zkCallback-156-thread-1-processing-n:127.0.0.1:38656_jbgv%2Foy) [n:127.0.0.1:38656_jbgv%2Foy    ] o.a.s.c.c.ZkStateReader Updated cluster state version to 1
   [junit4]   2> 284591 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:38656_jbgv%2Foy) [n:127.0.0.1:38656_jbgv%2Foy c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 284591 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:38656_jbgv%2Foy) [n:127.0.0.1:38656_jbgv%2Foy c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 284591 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:38656_jbgv%2Foy) [n:127.0.0.1:38656_jbgv%2Foy c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 284592 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:38656_jbgv%2Foy) [n:127.0.0.1:38656_jbgv%2Foy c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 284592 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:38656_jbgv%2Foy) [n:127.0.0.1:38656_jbgv%2Foy c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 284592 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:38656_jbgv%2Foy) [n:127.0.0.1:38656_jbgv%2Foy c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A108012B41E9020D-001/control-001/cores/collection1/'
   [junit4]   2> 284607 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:38656_jbgv%2Foy) [n:127.0.0.1:38656_jbgv%2Foy c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 284612 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:38656_jbgv%2Foy) [n:127.0.0.1:38656_jbgv%2Foy c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 284616 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:38656_jbgv%2Foy) [n:127.0.0.1:38656_jbgv%2Foy c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.0.0
   [junit4]   2> 284624 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:38656_jbgv%2Foy) [n:127.0.0.1:38656_jbgv%2Foy c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 284625 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:38656_jbgv%2Foy) [n:127.0.0.1:38656_jbgv%2Foy c:control_collection   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 284629 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:38656_jbgv%2Foy) [n:127.0.0.1:38656_jbgv%2Foy c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 284722 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:38656_jbgv%2Foy) [n:127.0.0.1:38656_jbgv%2Foy c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 284728 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:38656_jbgv%2Foy) [n:127.0.0.1:38656_jbgv%2Foy c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 284729 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:38656_jbgv%2Foy) [n:127.0.0.1:38656_jbgv%2Foy c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 284737 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:38656_jbgv%2Foy) [n:127.0.0.1:38656_jbgv%2Foy c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 284739 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:38656_jbgv%2Foy) [n:127.0.0.1:38656_jbgv%2Foy c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 284741 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:38656_jbgv%2Foy) [n:127.0.0.1:38656_jbgv%2Foy c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 284742 WARN  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:38656_jbgv%2Foy) [n:127.0.0.1:38656_jbgv%2Foy c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 284742 WARN  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:38656_jbgv%2Foy) [n:127.0.0.1:38656_jbgv%2Foy c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 284742 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:38656_jbgv%2Foy) [n:127.0.0.1:38656_jbgv%2Foy c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 284743 WARN  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:38656_jbgv%2Foy) [n:127.0.0.1:38656_jbgv%2Foy c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 284743 WARN  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:38656_jbgv%2Foy) [n:127.0.0.1:38656_jbgv%2Foy c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 284744 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:38656_jbgv%2Foy) [n:127.0.0.1:38656_jbgv%2Foy c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties
   [junit4]   2> org.apache.solr.core.SolrResourceNotFoundException: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1
   [junit4]   2> 	at org.apache.solr.cloud.ZkSolrResourceLoader.openResource(ZkSolrResourceLoader.java:99)
   [junit4]   2> 	at org.apache.solr.core.ConfigSetProperties.readFromResourceLoader(ConfigSetProperties.java:49)
   [junit4]   2> 	at org.apache.solr.core.ConfigSetService.createConfigSetProperties(ConfigSetService.java:114)
   [junit4]   2> 	at org.apache.solr.core.ConfigSetService.getConfig(ConfigSetService.java:76)
   [junit4]   2> 	at org.apache.solr.core.CoreContainer.create(CoreContainer.java:668)
   [junit4]   2> 	at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:397)
   [junit4]   2> 	at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:388)
   [junit4]   2> 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
   [junit4]   2> 	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:156)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 284745 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:38656_jbgv%2Foy) [n:127.0.0.1:38656_jbgv%2Foy c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 284745 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:38656_jbgv%2Foy) [n:127.0.0.1:38656_jbgv%2Foy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 284745 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:38656_jbgv%2Foy) [n:127.0.0.1:38656_jbgv%2Foy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A108012B41E9020D-001/control-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 284745 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:38656_jbgv%2Foy) [n:127.0.0.1:38656_jbgv%2Foy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@e909f2f
   [junit4]   2> 284746 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:38656_jbgv%2Foy) [n:127.0.0.1:38656_jbgv%2Foy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A108012B41E9020D-001/control-001/cores/collection1/data
   [junit4]   2> 284746 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:38656_jbgv%2Foy) [n:127.0.0.1:38656_jbgv%2Foy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A108012B41E9020D-001/control-001/cores/collection1/data/index/
   [junit4]   2> 284746 WARN  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:38656_jbgv%2Foy) [n:127.0.0.1:38656_jbgv%2Foy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A108012B41E9020D-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 284747 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:38656_jbgv%2Foy) [n:127.0.0.1:38656_jbgv%2Foy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A108012B41E9020D-001/control-001/cores/collection1/data/index
   [junit4]   2> 284747 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:38656_jbgv%2Foy) [n:127.0.0.1:38656_jbgv%2Foy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=18, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.21261105087057466]
   [junit4]   2> 284748 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:38656_jbgv%2Foy) [n:127.0.0.1:38656_jbgv%2Foy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@765e995a lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@22262a2e),segFN=segments_1,generation=1}
   [junit4]   2> 284748 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:38656_jbgv%2Foy) [n:127.0.0.1:38656_jbgv%2Foy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 284748 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:38656_jbgv%2Foy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A108012B41E9020D-001/control-001/cores/collection1/data/
   [junit4]   2> 284749 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:38656_jbgv%2Foy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A108012B41E9020D-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 284753 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:38656_jbgv%2Foy) [n:127.0.0.1:38656_jbgv%2Foy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 284754 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:38656_jbgv%2Foy) [n:127.0.0.1:38656_jbgv%2Foy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 284754 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:38656_jbgv%2Foy) [n:127.0.0.1:38656_jbgv%2Foy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 284754 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:38656_jbgv%2Foy) [n:127.0.0.1:38656_jbgv%2Foy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 284754 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:38656_jbgv%2Foy) [n:127.0.0.1:38656_jbgv%2Foy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 284754 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:38656_jbgv%2Foy) [n:127.0.0.1:38656_jbgv%2Foy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 284755 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:38656_jbgv%2Foy) [n:127.0.0.1:38656_jbgv%2Foy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 284755 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:38656_jbgv%2Foy) [n:127.0.0.1:38656_jbgv%2Foy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 284755 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:38656_jbgv%2Foy) [n:127.0.0.1:38656_jbgv%2Foy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 284757 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:38656_jbgv%2Foy) [n:127.0.0.1:38656_jbgv%2Foy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 284758 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:38656_jbgv%2Foy) [n:127.0.0.1:38656_jbgv%2Foy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 284759 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:38656_jbgv%2Foy) [n:127.0.0.1:38656_jbgv%2Foy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 284760 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:38656_jbgv%2Foy) [n:127.0.0.1:38656_jbgv%2Foy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 284764 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:38656_jbgv%2Foy) [n:127.0.0.1:38656_jbgv%2Foy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/mbeans,standard,/update/csv,/update/json/docs,/admin/luke,/admin/segments,/get,/admin/system,/replication,/admin/properties,/config,/schema,/admin/plugins,/admin/logging,/update/json,/admin/threads,/admin/ping,/update,/admin/file
   [junit4]   2> 284765 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:38656_jbgv%2Foy) [n:127.0.0.1:38656_jbgv%2Foy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 284765 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:38656_jbgv%2Foy) [n:127.0.0.1:38656_jbgv%2Foy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 284765 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:38656_jbgv%2Foy) [n:127.0.0.1:38656_jbgv%2Foy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 284766 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:38656_jbgv%2Foy) [n:127.0.0.1:38656_jbgv%2Foy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 284766 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:38656_jbgv%2Foy) [n:127.0.0.1:38656_jbgv%2Foy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 284767 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:38656_jbgv%2Foy) [n:127.0.0.1:38656_jbgv%2Foy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=43, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
   [junit4]   2> 284770 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:38656_jbgv%2Foy) [n:127.0.0.1:38656_jbgv%2Foy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@765e995a lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@22262a2e),segFN=segments_1,generation=1}
   [junit4]   2> 284770 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:38656_jbgv%2Foy) [n:127.0.0.1:38656_jbgv%2Foy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 284770 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:38656_jbgv%2Foy) [n:127.0.0.1:38656_jbgv%2Foy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@6c8167a[collection1] main
   [junit4]   2> 284771 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:38656_jbgv%2Foy) [n:127.0.0.1:38656_jbgv%2Foy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 284771 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:38656_jbgv%2Foy) [n:127.0.0.1:38656_jbgv%2Foy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 284771 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:38656_jbgv%2Foy) [n:127.0.0.1:38656_jbgv%2Foy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 284772 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:38656_jbgv%2Foy) [n:127.0.0.1:38656_jbgv%2Foy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 284772 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:38656_jbgv%2Foy) [n:127.0.0.1:38656_jbgv%2Foy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 284772 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:38656_jbgv%2Foy) [n:127.0.0.1:38656_jbgv%2Foy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 284772 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:38656_jbgv%2Foy) [n:127.0.0.1:38656_jbgv%2Foy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 284773 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:38656_jbgv%2Foy) [n:127.0.0.1:38656_jbgv%2Foy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 284773 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:38656_jbgv%2Foy) [n:127.0.0.1:38656_jbgv%2Foy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 284773 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:38656_jbgv%2Foy) [n:127.0.0.1:38656_jbgv%2Foy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 284774 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:38656_jbgv%2Foy) [n:127.0.0.1:38656_jbgv%2Foy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 284774 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:38656_jbgv%2Foy) [n:127.0.0.1:38656_jbgv%2Foy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 284774 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:38656_jbgv%2Foy) [n:127.0.0.1:38656_jbgv%2Foy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 284774 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:38656_jbgv%2Foy) [n:127.0.0.1:38656_jbgv%2Foy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1506738866605785088
   [junit4]   2> 284776 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:38656_jbgv%2Foy) [n:127.0.0.1:38656_jbgv%2Foy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 2 ms to seed version buckets with highest version 1506738866605785088
   [junit4]   2> 284776 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:38656_jbgv%2Foy) [n:127.0.0.1:38656_jbgv%2Foy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 284777 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:38656_jbgv%2Foy) [n:127.0.0.1:38656_jbgv%2Foy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 284778 INFO  (searcherExecutor-675-thread-1-processing-n:127.0.0.1:38656_jbgv%2Foy x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:38656_jbgv%2Foy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@6c8167a[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 284778 INFO  (coreZkRegister-668-thread-1-processing-n:127.0.0.1:38656_jbgv%2Foy x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:38656_jbgv%2Foy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:https://127.0.0.1:38656/jbgv/oy collection:control_collection shard:shard1
   [junit4]   2> 284779 INFO  (coreZkRegister-668-thread-1-processing-n:127.0.0.1:38656_jbgv%2Foy x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:38656_jbgv%2Foy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 284783 INFO  (coreZkRegister-668-thread-1-processing-n:127.0.0.1:38656_jbgv%2Foy x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:38656_jbgv%2Foy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94171179046076420-core_node1-n_0000000000
   [junit4]   2> 284784 INFO  (coreZkRegister-668-thread-1-processing-n:127.0.0.1:38656_jbgv%2Foy x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:38656_jbgv%2Foy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 284785 INFO  (zkCallback-156-thread-1-processing-n:127.0.0.1:38656_jbgv%2Foy) [n:127.0.0.1:38656_jbgv%2Foy    ] o.a.s.c.DistributedQueue NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 284785 INFO  (coreZkRegister-668-thread-1-processing-n:127.0.0.1:38656_jbgv%2Foy x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:38656_jbgv%2Foy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 284786 INFO  (coreZkRegister-668-thread-1-processing-n:127.0.0.1:38656_jbgv%2Foy x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:38656_jbgv%2Foy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 284786 INFO  (coreZkRegister-668-thread-1-processing-n:127.0.0.1:38656_jbgv%2Foy x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:38656_jbgv%2Foy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:38656/jbgv/oy/collection1/
   [junit4]   2> 284786 INFO  (OverseerStateUpdate-94171179046076420-127.0.0.1:38656_jbgv%2Foy-n_0000000000) [n:127.0.0.1:38656_jbgv%2Foy    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 284786 INFO  (coreZkRegister-668-thread-1-processing-n:127.0.0.1:38656_jbgv%2Foy x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:38656_jbgv%2Foy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 284786 INFO  (coreZkRegister-668-thread-1-processing-n:127.0.0.1:38656_jbgv%2Foy x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:38656_jbgv%2Foy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy https://127.0.0.1:38656/jbgv/oy/collection1/ has no replicas
   [junit4]   2> 284786 INFO  (coreZkRegister-668-thread-1-processing-n:127.0.0.1:38656_jbgv%2Foy x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:38656_jbgv%2Foy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:38656/jbgv/oy/collection1/ shard1
   [junit4]   2> 284787 INFO  (coreZkRegister-668-thread-1-processing-n:127.0.0.1:38656_jbgv%2Foy x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:38656_jbgv%2Foy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 284790 INFO  (zkCallback-156-thread-1-processing-n:127.0.0.1:38656_jbgv%2Foy) [n:127.0.0.1:38656_jbgv%2Foy    ] o.a.s.c.DistributedQueue NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 284791 INFO  (OverseerStateUpdate-94171179046076420-127.0.0.1:38656_jbgv%2Foy-n_0000000000) [n:127.0.0.1:38656_jbgv%2Foy    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"https://127.0.0.1:38656/jbgv/oy",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 1
   [junit4]   2> 284894 INFO  (zkCallback-156-thread-1-processing-n:127.0.0.1:38656_jbgv%2Foy) [n:127.0.0.1:38656_jbgv%2Foy    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 284896 INFO  (zkCallback-156-thread-1-processing-n:127.0.0.1:38656_jbgv%2Foy) [n:127.0.0.1:38656_jbgv%2Foy    ] o.a.s.c.c.ZkStateReader Updated cluster state version to 2
   [junit4]   2> 284941 INFO  (coreZkRegister-668-thread-1-processing-n:127.0.0.1:38656_jbgv%2Foy x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:38656_jbgv%2Foy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are https://127.0.0.1:38656/jbgv/oy/collection1/ and leader is https://127.0.0.1:38656/jbgv/oy/collection1/
   [junit4]   2> 284941 INFO  (coreZkRegister-668-thread-1-processing-n:127.0.0.1:38656_jbgv%2Foy x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:38656_jbgv%2Foy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:38656/jbgv/oy
   [junit4]   2> 284941 INFO  (coreZkRegister-668-thread-1-processing-n:127.0.0.1:38656_jbgv%2Foy x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:38656_jbgv%2Foy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 284942 INFO  (coreZkRegister-668-thread-1-processing-n:127.0.0.1:38656_jbgv%2Foy x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:38656_jbgv%2Foy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 284942 INFO  (coreZkRegister-668-thread-1-processing-n:127.0.0.1:38656_jbgv%2Foy x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:38656_jbgv%2Foy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 284943 INFO  (zkCallback-156-thread-1-processing-n:127.0.0.1:38656_jbgv%2Foy) [n:127.0.0.1:38656_jbgv%2Foy    ] o.a.s.c.DistributedQueue NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 284944 INFO  (OverseerStateUpdate-94171179046076420-127.0.0.1:38656_jbgv%2Foy-n_0000000000) [n:127.0.0.1:38656_jbgv%2Foy    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:38656/jbgv/oy",
   [junit4]   2>   "node_name":"127.0.0.1:38656_jbgv%2Foy",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"} current state version: 2
   [junit4]   2> 284945 INFO  (OverseerStateUpdate-94171179046076420-127.0.0.1:38656_jbgv%2Foy-n_0000000000) [n:127.0.0.1:38656_jbgv%2Foy    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:38656/jbgv/oy",
   [junit4]   2>   "node_name":"127.0.0.1:38656_jbgv%2Foy",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 285048 INFO  (zkCallback-156-thread-1-processing-n:127.0.0.1:38656_jbgv%2Foy) [n:127.0.0.1:38656_jbgv%2Foy    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 285049 INFO  (zkCallback-156-thread-1-processing-n:127.0.0.1:38656_jbgv%2Foy) [n:127.0.0.1:38656_jbgv%2Foy    ] o.a.s.c.c.ZkStateReader Updated cluster state version to 3
   [junit4]   2> 285099 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 285100 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 285101 INFO  (zkCallback-158-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@e453083 name:ZooKeeperConnection Watcher:127.0.0.1:58829/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 285102 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 285102 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 285102 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 285104 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 285474 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A108012B41E9020D-001/shard-1-001/cores/collection1
   [junit4]   2> 285475 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A108012B41E9020D-001/shard-1-001
   [junit4]   2> 285476 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [    ] o.e.j.s.Server jetty-9.2.11.v20150529
   [junit4]   2> 285478 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@417186fb{/jbgv/oy,null,AVAILABLE}
   [junit4]   2> 285482 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [    ] o.e.j.s.ServerConnector Started ServerConnector@314abee8{SSL-http/1.1}{127.0.0.1:50779}
   [junit4]   2> 285482 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [    ] o.e.j.s.Server Started @289093ms
   [junit4]   2> 285482 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A108012B41E9020D-001/tempDir-001/jetty1, solrconfig=solrconfig.xml, hostContext=/jbgv/oy, hostPort=50779, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A108012B41E9020D-001/shard-1-001/cores}
   [junit4]   2> 285484 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@c387f44
   [junit4]   2> 285484 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A108012B41E9020D-001/shard-1-001/'
   [junit4]   2> 285499 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A108012B41E9020D-001/shard-1-001/solr.xml
   [junit4]   2> 285505 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A108012B41E9020D-001/shard-1-001/cores
   [junit4]   2> 285505 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [    ] o.a.s.c.CoreContainer New CoreContainer 730521635
   [junit4]   2> 285505 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A108012B41E9020D-001/shard-1-001/]
   [junit4]   2> 285505 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A108012B41E9020D-001/shard-1-001/lib
   [junit4]   2> 285505 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A108012B41E9020D-001/shard-1-001/lib).
   [junit4]   2> 285512 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 285515 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 285522 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 285522 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 285523 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:58829/solr
   [junit4]   2> 285523 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 285523 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 285552 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 285559 INFO  (zkCallback-160-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6a5930d7 name:ZooKeeperConnection Watcher:127.0.0.1:58829 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 285559 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 285560 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 285562 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [n:127.0.0.1:50779_jbgv%2Foy    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 285567 INFO  (zkCallback-161-thread-1-processing-n:127.0.0.1:50779_jbgv%2Foy) [n:127.0.0.1:50779_jbgv%2Foy    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2b6df771 name:ZooKeeperConnection Watcher:127.0.0.1:58829/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 285567 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [n:127.0.0.1:50779_jbgv%2Foy    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 285572 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [n:127.0.0.1:50779_jbgv%2Foy    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 286576 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [n:127.0.0.1:50779_jbgv%2Foy    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50779_jbgv%2Foy
   [junit4]   2> 286576 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [n:127.0.0.1:50779_jbgv%2Foy    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:50779_jbgv%2Foy
   [junit4]   2> 286579 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [n:127.0.0.1:50779_jbgv%2Foy    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 286580 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [n:127.0.0.1:50779_jbgv%2Foy    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94171179046076423-127.0.0.1:50779_jbgv%2Foy-n_0000000001
   [junit4]   2> 286580 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [n:127.0.0.1:50779_jbgv%2Foy    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/94171179046076420-127.0.0.1:38656_jbgv%2Foy-n_0000000000 to know if I could be the leader
   [junit4]   2> 286581 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [n:127.0.0.1:50779_jbgv%2Foy    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 286581 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [n:127.0.0.1:50779_jbgv%2Foy    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 286582 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [n:127.0.0.1:50779_jbgv%2Foy    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A108012B41E9020D-001/shard-1-001/cores
   [junit4]   2> 286583 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [n:127.0.0.1:50779_jbgv%2Foy    ] o.a.s.c.SolrCore Created CoreDescriptor: {name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, loadOnStartup=true, configSetProperties=configsetprops.json, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A108012B41E9020D-001/shard-1-001/cores/collection1, collection=collection1, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A108012B41E9020D-001/shard-1-001/cores/collection1/, coreNodeName=, dataDir=data/, shard=}
   [junit4]   2> 286583 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [n:127.0.0.1:50779_jbgv%2Foy    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A108012B41E9020D-001/shard-1-001/cores/collection1/
   [junit4]   2> 286583 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [n:127.0.0.1:50779_jbgv%2Foy    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 286584 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [n:127.0.0.1:50779_jbgv%2Foy    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1
   [junit4]   2> 286584 INFO  (coreLoadExecutor-685-thread-1-processing-n:127.0.0.1:50779_jbgv%2Foy) [n:127.0.0.1:50779_jbgv%2Foy c:collection1   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 286584 INFO  (coreLoadExecutor-685-thread-1-processing-n:127.0.0.1:50779_jbgv%2Foy) [n:127.0.0.1:50779_jbgv%2Foy c:collection1   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 286584 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A108012B41E9020D]) [n:127.0.0.1:50779_jbgv%2Foy    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 286585 INFO  (zkCallback-156-thread-1-processing-n:127.0.0.1:38656_jbgv%2Foy) [n:127.0.0.1:38656_jbgv%2Foy    ] o.a.s.c.DistributedQueue NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 286585 INFO  (coreLoadExecutor-685-thread-1-processing-n:127.0.0.1:50779_jbgv%2Foy) [n:127.0.0.1:50779_jbgv%2Foy c:collection1   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 286586 INFO  (OverseerStateUpdate-94171179046076420-127.0.0.1:38656_jbgv%2Foy-n_0000000000) [n:127.0.0.1:38656_jbgv%2Foy    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:50779/jbgv/oy",
   [junit4]   2>   "node_name":"127.0.0.1:50779_jbgv%2Foy",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "operation":"state"} current state version: 3
   [junit4]   2> 286587 INFO  (OverseerStateUpdate-94171179046076420-127.0.0.1:38656_jbgv%2Foy-n_0000000000) [n:127.0.0.1:38656_jbgv%2Foy    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:50779/jbgv/oy",
   [junit4]   2>   "node_name":"127.0.0.1:50779_jbgv%2Foy",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 286587 INFO  (OverseerStateUpdate-94171179046076420-127.0.0.1:38656_jbgv%2Foy-n_0000000000) [n:127.0.0.1:38656_jbgv%2Foy    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 286587 INFO  (OverseerStateUpdate-94171179046076420-127.0.0.1:38656_jbgv%2Foy-n_0000000000) [n:127.0.0.1:38656_jbgv%2Foy    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2
   [junit4]   2> 286588 INFO  (zkCallback-161-thread-1-processing-n:127.0.0.1:50779_jbgv%2Foy) [n:127.0.0.1:50779_jbgv%2Foy    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 286588 INFO  (zkCallback-158-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 286589 INFO  (zkCallback-156-thread-1-processing-n:127.0.0.1:38656_jbgv%2Foy) [n:127.0.0.1:38656_jbgv%2Foy    ] o.a.s.c.c.ZkStateReader A cluster state c

[...truncated too long message...]

org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1720)
   [junit4]   2> 	at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1699)
   [junit4]   2> 	at org.apache.solr.cloud.hdfs.HdfsTestUtil.teardownClass(HdfsTestUtil.java:197)
   [junit4]   2> 	at org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.teardownClass(HdfsCollectionsAPIDistributedZkTest.java:50)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   [junit4]   2> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2> 	at java.lang.reflect.Method.invoke(Method.java:483)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:799)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> Caused by: java.lang.NullPointerException
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.blockmanagement.BlocksMap.size(BlocksMap.java:198)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.getTotalBlocks(BlockManager.java:3291)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlocksTotal(FSNamesystem.java:6223)
   [junit4]   2> 	... 53 more
   [junit4]   2> 4703419 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[A108012B41E9020D]-worker) [n:127.0.0.1:41345_zqga%2Fv c:addReplicaColl s:shard2 r:core_node6 x:addReplicaColl_shard2_replica3] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> Jul 15, 2015 2:42:57 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 2 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene53): {other_tl1=PostingsFormat(name=LuceneVarGapFixedInterval), range_facet_l_dv=PostingsFormat(name=Memory doPackFST= true), _version_=PostingsFormat(name=LuceneVarGapFixedInterval), multiDefault=PostingsFormat(name=LuceneVarGapDocFreqInterval), a_t=PostingsFormat(name=LuceneVarGapFixedInterval), intDefault=PostingsFormat(name=LuceneVarGapFixedInterval), a_i1=PostingsFormat(name=LuceneVarGapFixedInterval), id=PostingsFormat(name=Memory doPackFST= true), range_facet_i_dv=PostingsFormat(name=LuceneVarGapFixedInterval), text=FST50, range_facet_l=PostingsFormat(name=LuceneVarGapFixedInterval), timestamp=PostingsFormat(name=LuceneVarGapFixedInterval)}, docValues:{range_facet_l_dv=DocValuesFormat(name=Memory), range_facet_i_dv=DocValuesFormat(name=Lucene50), timestamp=DocValuesFormat(name=Lucene50)}, sim=DefaultSimilarity, locale=ar_TN, timezone=Asia/Yakutsk
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.8.0_25 (64-bit)/cpus=4,threads=2,free=75500376,total=525336576
   [junit4]   2> NOTE: All tests run in this JVM: [HighlighterTest, CollectionReloadTest, MinimalSchemaTest, TestFreeTextSuggestions, TestSolrIndexConfig, QueryElevationComponentTest, TestSmileRequest, TestSolrDynamicMBean, DefaultValueUpdateProcessorTest, SimplePostToolTest, DistributedFacetPivotSmallAdvancedTest, TestNonDefinedSimilarityFactory, TestSweetSpotSimilarityFactory, SuggesterFSTTest, FacetPivotSmallTest, TestDefaultSearchFieldResource, TestRandomFaceting, TestStandardQParsers, UUIDFieldTest, ExitableDirectoryReaderTest, TestDynamicFieldResource, TestRemoteStreaming, TestInfoStreamLogging, MBeansHandlerTest, AssignTest, EnumFieldTest, TestCollapseQParserPlugin, TestSolrDeletionPolicy1, SolrIndexConfigTest, TestBlendedInfixSuggestions, TestRestoreCore, TestCodecSupport, SegmentsInfoRequestHandlerTest, TestFileDictionaryLookup, FastVectorHighlighterTest, AsyncCallRequestStatusResponseTest, TestSolrConfigHandler, SignatureUpdateProcessorFactoryTest, RequestHandlersTest, ParsingFieldUpdateProcessorsTest, TestJoin, LeaderElectionIntegrationTest, PeerSyncTest, BasicZkTest, TestStressLucene, DebugComponentTest, TestBinaryField, TestRangeQuery, ZkNodePropsTest, TestSimpleTrackingShardHandler, TestRealTimeGet, TestRTimer, AddBlockUpdateTest, ResponseHeaderTest, DistributedQueueTest, DistributedFacetPivotWhiteBoxTest, TestReRankQParserPlugin, DeleteInactiveReplicaTest, RollingRestartTest, TestDistributedSearch, TestCloudManagedSchemaConcurrent, URLClassifyProcessorTest, HdfsDirectoryTest, TestReplicationHandler, TestLazyCores, DistributedFacetPivotLargeTest, SolrRequestParserTest, SyncSliceTest, TestFieldResource, TestConfigReload, LeaderFailoverAfterPartitionTest, TestSolrConfigHandlerConcurrent, DataDrivenBlockJoinTest, PrimitiveFieldTypeTest, ConnectionManagerTest, ExternalFileFieldSortTest, TestLeaderElectionZkExpiry, TestWordDelimiterFilterFactory, TestFunctionQuery, TestConfig, TestJsonFacets, TestSolrQueryParserResource, TestFieldTypeCollectionResource, TestEmbeddedSolrServerConstructors, TestOrdValues, FullSolrCloudDistribCmdsTest, LeaderElectionTest, ZkCLITest, AutoCommitTest, TestDistributedGrouping, TestFaceting, TestHashPartitioner, DistributedSpellCheckComponentTest, HardAutoCommitTest, TestCoreContainer, SolrCoreTest, StatsComponentTest, SolrCmdDistributorTest, BasicFunctionalityTest, TestIndexSearcher, SoftAutoCommitTest, CoreAdminHandlerTest, TestTrie, TestCSVLoader, SchemaVersionSpecificBehaviorTest, TestUpdate, TestAtomicUpdateErrorCases, StatelessScriptUpdateProcessorFactoryTest, TestSurroundQueryParser, DisMaxRequestHandlerTest, TestQueryUtils, TestWriterPerf, FileBasedSpellCheckerTest, TestValueSourceCache, TestIndexingPerformance, RequiredFieldsTest, LoggingHandlerTest, TestJmxIntegration, UpdateRequestProcessorFactoryTest, QueryParsingTest, UniqFieldsUpdateProcessorFactoryTest, CSVRequestHandlerTest, TestBinaryResponseWriter, SOLR749Test, UpdateParamsTest, BadComponentTest, MultiTermTest, TestSearchPerf, TestElisionMultitermQuery, NotRequiredUniqueKeyTest, TestPhraseSuggestions, SynonymTokenizerTest, TestXIncludeConfig, TestLMJelinekMercerSimilarityFactory, TestDefaultSimilarityFactory, TimeZoneUtilsTest, ScriptEngineTest, OpenExchangeRatesOrgProviderTest, ChaosMonkeyNothingIsSafeTest, CollectionTooManyReplicasTest, DistribCursorPagingTest, DistributedVersionInfoTest, MultiThreadedOCPTest, OverseerStatusTest, SSLMigrationTest, SaslZkACLProviderTest, SharedFSAutoReplicaFailoverTest, HdfsCollectionsAPIDistributedZkTest]
   [junit4] Completed [517/517] on J0 in 1084.14s, 1 test, 1 error <<< FAILURES!

[...truncated 13 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:533: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:474: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:61: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/extra-targets.xml:39: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build.xml:230: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/common-build.xml:513: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/common-build.xml:1426: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/common-build.xml:981: There were test failures: 517 suites, 2032 tests, 2 errors, 1 failure, 20 ignored (10 assumptions)

Total time: 194 minutes 47 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
No prior successful build to compare, so performing full copy of artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



Re: [JENKINS] Lucene-Solr-NightlyTests-trunk - Build # 739 - Still Failing

Posted by Michael McCandless <lu...@mikemccandless.com>.
I just committed a fix.  This was caused by a new assert from
LUCENE-6616, basically making IndexWriter more picky such that a codec
is not allowed to have created a file if in fact it didn't create it.

In this case the SimpleTextSIFormat was claiming to have created a
file (si.addFile) before createOutput succeeded, but then createOutput
hit an exc (and the Directory impl took care of removing the file
itself)...

Mike McCandless

http://blog.mikemccandless.com


On Tue, Jul 14, 2015 at 5:30 AM, Michael McCandless
<lu...@mikemccandless.com> wrote:
> I'll dig, this is a new assertion I added recently to IFD.
>
> Mike McCandless
>
> http://blog.mikemccandless.com
>
>
> On Tue, Jul 14, 2015 at 12:04 AM, Apache Jenkins Server
> <je...@builds.apache.org> wrote:
>> Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-trunk/739/
>>
>> 1 tests failed.
>> REGRESSION:  org.apache.lucene.codecs.simpletext.TestSimpleTextFieldInfoFormat.testRandomExceptions
>>
>> Error Message:
>> hit unexpected FileNotFoundException: file=_3w.si
>>
>> Stack Trace:
>> java.lang.AssertionError: hit unexpected FileNotFoundException: file=_3w.si
>>         at __randomizedtesting.SeedInfo.seed([A1871D12B75F5C9E:C9A873D62951053E]:0)
>>         at org.apache.lucene.index.IndexFileDeleter.deleteFile(IndexFileDeleter.java:751)
>>         at org.apache.lucene.index.IndexFileDeleter.deletePendingFiles(IndexFileDeleter.java:528)
>>         at org.apache.lucene.index.IndexFileDeleter.decRef(IndexFileDeleter.java:636)
>>         at org.apache.lucene.index.IndexFileDeleter.checkpoint(IndexFileDeleter.java:578)
>>         at org.apache.lucene.index.IndexWriter.checkpoint(IndexWriter.java:2298)
>>         at org.apache.lucene.index.IndexWriter.publishFlushedSegment(IndexWriter.java:2356)
>>         at org.apache.lucene.index.DocumentsWriterFlushQueue$FlushTicket.publishFlushedSegment(DocumentsWriterFlushQueue.java:198)
>>         at org.apache.lucene.index.DocumentsWriterFlushQueue$FlushTicket.finishFlush(DocumentsWriterFlushQueue.java:213)
>>         at org.apache.lucene.index.DocumentsWriterFlushQueue$SegmentFlushTicket.publish(DocumentsWriterFlushQueue.java:249)
>>         at org.apache.lucene.index.DocumentsWriterFlushQueue.innerPurge(DocumentsWriterFlushQueue.java:116)
>>         at org.apache.lucene.index.DocumentsWriterFlushQueue.forcePurge(DocumentsWriterFlushQueue.java:138)
>>         at org.apache.lucene.index.DocumentsWriter.flushAllThreads(DocumentsWriter.java:632)
>>         at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:422)
>>         at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:86)
>>         at org.apache.lucene.index.BaseIndexFileFormatTestCase.testRandomExceptions(BaseIndexFileFormatTestCase.java:457)
>>         at org.apache.lucene.index.BaseFieldInfoFormatTestCase.testRandomExceptions(BaseFieldInfoFormatTestCase.java:50)
>>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>         at java.lang.reflect.Method.invoke(Method.java:483)
>>         at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
>>         at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
>>         at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
>>         at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
>>         at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
>>         at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
>>         at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
>>         at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
>>         at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
>>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
>>         at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
>>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
>>         at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
>>         at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
>>         at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
>>         at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
>>         at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
>>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>         at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
>>         at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
>>         at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
>>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>         at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
>>         at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
>>         at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
>>         at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
>>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
>>         at java.lang.Thread.run(Thread.java:745)
>>
>>
>>
>>
>> Build Log:
>> [...truncated 6446 lines...]
>>    [junit4] Suite: org.apache.lucene.codecs.simpletext.TestSimpleTextFieldInfoFormat
>>    [junit4]   1> Unexpected exception: dumping fake-exception-log:...
>>    [junit4]   1>
>>    [junit4]   1> TEST: got expected fake exc:a random IOException (_7.si)
>>    [junit4]   1> java.io.IOException: a random IOException (_7.si)
>>    [junit4]   1>        at org.apache.lucene.store.MockDirectoryWrapper.maybeThrowIOExceptionOnOpen(MockDirectoryWrapper.java:456)
>>    [junit4]   1>        at org.apache.lucene.store.MockDirectoryWrapper.createOutput(MockDirectoryWrapper.java:548)
>>    [junit4]   1>        at org.apache.lucene.store.LockValidatingDirectoryWrapper.createOutput(LockValidatingDirectoryWrapper.java:44)
>>    [junit4]   1>        at org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:43)
>>    [junit4]   1>        at org.apache.lucene.codecs.simpletext.SimpleTextSegmentInfoFormat.write(SimpleTextSegmentInfoFormat.java:159)
>>    [junit4]   1>        at org.apache.lucene.index.DocumentsWriterPerThread.sealFlushedSegment(DocumentsWriterPerThread.java:502)
>>    [junit4]   1>        at org.apache.lucene.index.DocumentsWriterPerThread.flush(DocumentsWriterPerThread.java:460)
>>    [junit4]   1>        at org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:510)
>>    [junit4]   1>        at org.apache.lucene.index.DocumentsWriter.flushAllThreads(DocumentsWriter.java:622)
>>    [junit4]   1>        at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:422)
>>    [junit4]   1>        at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:86)
>>    [junit4]   1>        at org.apache.lucene.index.BaseIndexFileFormatTestCase.testRandomExceptions(BaseIndexFileFormatTestCase.java:457)
>>    [junit4]   1>        at org.apache.lucene.index.BaseFieldInfoFormatTestCase.testRandomExceptions(BaseFieldInfoFormatTestCase.java:50)
>>    [junit4]   1>        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>    [junit4]   1>        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>>    [junit4]   1>        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>    [junit4]   1>        at java.lang.reflect.Method.invoke(Method.java:483)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
>>    [junit4]   1>        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
>>    [junit4]   1>        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
>>    [junit4]   1>        at java.lang.Thread.run(Thread.java:745)
>>    [junit4]   1>
>>    [junit4]   1> TEST: got expected fake exc:a random IOException (_a.pst)
>>    [junit4]   1> java.io.IOException: a random IOException (_a.pst)
>>    [junit4]   1>        at org.apache.lucene.store.MockDirectoryWrapper.maybeThrowIOExceptionOnOpen(MockDirectoryWrapper.java:456)
>>    [junit4]   1>        at org.apache.lucene.store.MockDirectoryWrapper.openInput(MockDirectoryWrapper.java:635)
>>    [junit4]   1>        at org.apache.lucene.store.FilterDirectory.openInput(FilterDirectory.java:89)
>>    [junit4]   1>        at org.apache.lucene.store.FilterDirectory.openInput(FilterDirectory.java:89)
>>    [junit4]   1>        at org.apache.lucene.store.FilterDirectory.openInput(FilterDirectory.java:89)
>>    [junit4]   1>        at org.apache.lucene.codecs.simpletext.SimpleTextCompoundFormat.write(SimpleTextCompoundFormat.java:179)
>>    [junit4]   1>        at org.apache.lucene.index.IndexWriter.createCompoundFile(IndexWriter.java:4652)
>>    [junit4]   1>        at org.apache.lucene.index.DocumentsWriterPerThread.sealFlushedSegment(DocumentsWriterPerThread.java:493)
>>    [junit4]   1>        at org.apache.lucene.index.DocumentsWriterPerThread.flush(DocumentsWriterPerThread.java:460)
>>    [junit4]   1>        at org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:510)
>>    [junit4]   1>        at org.apache.lucene.index.DocumentsWriter.flushAllThreads(DocumentsWriter.java:622)
>>    [junit4]   1>        at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:422)
>>    [junit4]   1>        at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:86)
>>    [junit4]   1>        at org.apache.lucene.index.BaseIndexFileFormatTestCase.testRandomExceptions(BaseIndexFileFormatTestCase.java:457)
>>    [junit4]   1>        at org.apache.lucene.index.BaseFieldInfoFormatTestCase.testRandomExceptions(BaseFieldInfoFormatTestCase.java:50)
>>    [junit4]   1>        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>    [junit4]   1>        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>>    [junit4]   1>        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>    [junit4]   1>        at java.lang.reflect.Method.invoke(Method.java:483)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
>>    [junit4]   1>        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
>>    [junit4]   1>        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
>>    [junit4]   1>        at java.lang.Thread.run(Thread.java:745)
>>    [junit4]   1>
>>    [junit4]   1> TEST: got expected fake exc:a random IOException (_e.fld)
>>    [junit4]   1> java.io.FileNotFoundException: a random IOException (_e.fld)
>>    [junit4]   1>        at org.apache.lucene.store.MockDirectoryWrapper.maybeThrowIOExceptionOnOpen(MockDirectoryWrapper.java:458)
>>    [junit4]   1>        at org.apache.lucene.store.MockDirectoryWrapper.openInput(MockDirectoryWrapper.java:635)
>>    [junit4]   1>        at org.apache.lucene.codecs.simpletext.SimpleTextStoredFieldsReader.<init>(SimpleTextStoredFieldsReader.java:68)
>>    [junit4]   1>        at org.apache.lucene.codecs.simpletext.SimpleTextStoredFieldsFormat.fieldsReader(SimpleTextStoredFieldsFormat.java:40)
>>    [junit4]   1>        at org.apache.lucene.index.SegmentCoreReaders.<init>(SegmentCoreReaders.java:117)
>>    [junit4]   1>        at org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:65)
>>    [junit4]   1>        at org.apache.lucene.index.ReadersAndUpdates.getReader(ReadersAndUpdates.java:145)
>>    [junit4]   1>        at org.apache.lucene.index.BufferedUpdatesStream$SegmentState.<init>(BufferedUpdatesStream.java:378)
>>    [junit4]   1>        at org.apache.lucene.index.BufferedUpdatesStream.openSegmentStates(BufferedUpdatesStream.java:410)
>>    [junit4]   1>        at org.apache.lucene.index.BufferedUpdatesStream.applyDeletesAndUpdates(BufferedUpdatesStream.java:255)
>>    [junit4]   1>        at org.apache.lucene.index.IndexWriter._mergeInit(IndexWriter.java:3820)
>>    [junit4]   1>        at org.apache.lucene.index.IndexWriter.mergeInit(IndexWriter.java:3778)
>>    [junit4]   1>        at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3630)
>>    [junit4]   1>        at org.apache.lucene.index.SerialMergeScheduler.merge(SerialMergeScheduler.java:40)
>>    [junit4]   1>        at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:1912)
>>    [junit4]   1>        at org.apache.lucene.index.IndexWriter.doAfterSegmentFlushed(IndexWriter.java:4703)
>>    [junit4]   1>        at org.apache.lucene.index.DocumentsWriter$MergePendingEvent.process(DocumentsWriter.java:697)
>>    [junit4]   1>        at org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:4729)
>>    [junit4]   1>        at org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:4720)
>>    [junit4]   1>        at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1459)
>>    [junit4]   1>        at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1237)
>>    [junit4]   1>        at org.apache.lucene.index.BaseIndexFileFormatTestCase.testRandomExceptions(BaseIndexFileFormatTestCase.java:427)
>>    [junit4]   1>        at org.apache.lucene.index.BaseFieldInfoFormatTestCase.testRandomExceptions(BaseFieldInfoFormatTestCase.java:50)
>>    [junit4]   1>        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>    [junit4]   1>        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>>    [junit4]   1>        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>    [junit4]   1>        at java.lang.reflect.Method.invoke(Method.java:483)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
>>    [junit4]   1>        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
>>    [junit4]   1>        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
>>    [junit4]   1>        at java.lang.Thread.run(Thread.java:745)
>>    [junit4]   1>
>>    [junit4]   1> TEST: got expected fake exc:a random IOException (_n.inf)
>>    [junit4]   1> java.io.IOException: a random IOException (_n.inf)
>>    [junit4]   1>        at org.apache.lucene.store.MockDirectoryWrapper.maybeThrowIOExceptionOnOpen(MockDirectoryWrapper.java:456)
>>    [junit4]   1>        at org.apache.lucene.store.MockDirectoryWrapper.openInput(MockDirectoryWrapper.java:635)
>>    [junit4]   1>        at org.apache.lucene.store.Directory.openChecksumInput(Directory.java:109)
>>    [junit4]   1>        at org.apache.lucene.store.MockDirectoryWrapper.openChecksumInput(MockDirectoryWrapper.java:1009)
>>    [junit4]   1>        at org.apache.lucene.codecs.simpletext.SimpleTextFieldInfosFormat.read(SimpleTextFieldInfosFormat.java:71)
>>    [junit4]   1>        at org.apache.lucene.index.SegmentCoreReaders.<init>(SegmentCoreReaders.java:99)
>>    [junit4]   1>        at org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:65)
>>    [junit4]   1>        at org.apache.lucene.index.ReadersAndUpdates.getReader(ReadersAndUpdates.java:145)
>>    [junit4]   1>        at org.apache.lucene.index.BufferedUpdatesStream$SegmentState.<init>(BufferedUpdatesStream.java:378)
>>    [junit4]   1>        at org.apache.lucene.index.BufferedUpdatesStream.openSegmentStates(BufferedUpdatesStream.java:410)
>>    [junit4]   1>        at org.apache.lucene.index.BufferedUpdatesStream.applyDeletesAndUpdates(BufferedUpdatesStream.java:255)
>>    [junit4]   1>        at org.apache.lucene.index.IndexWriter._mergeInit(IndexWriter.java:3820)
>>    [junit4]   1>        at org.apache.lucene.index.IndexWriter.mergeInit(IndexWriter.java:3778)
>>    [junit4]   1>        at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3630)
>>    [junit4]   1>        at org.apache.lucene.index.SerialMergeScheduler.merge(SerialMergeScheduler.java:40)
>>    [junit4]   1>        at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:1912)
>>    [junit4]   1>        at org.apache.lucene.index.IndexWriter.doAfterSegmentFlushed(IndexWriter.java:4703)
>>    [junit4]   1>        at org.apache.lucene.index.DocumentsWriter$MergePendingEvent.process(DocumentsWriter.java:697)
>>    [junit4]   1>        at org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:4729)
>>    [junit4]   1>        at org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:4720)
>>    [junit4]   1>        at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1459)
>>    [junit4]   1>        at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1237)
>>    [junit4]   1>        at org.apache.lucene.index.BaseIndexFileFormatTestCase.testRandomExceptions(BaseIndexFileFormatTestCase.java:427)
>>    [junit4]   1>        at org.apache.lucene.index.BaseFieldInfoFormatTestCase.testRandomExceptions(BaseFieldInfoFormatTestCase.java:50)
>>    [junit4]   1>        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>    [junit4]   1>        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>>    [junit4]   1>        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>    [junit4]   1>        at java.lang.reflect.Method.invoke(Method.java:483)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
>>    [junit4]   1>        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
>>    [junit4]   1>        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
>>    [junit4]   1>        at java.lang.Thread.run(Thread.java:745)
>>    [junit4]   1>
>>    [junit4]   1> TEST: got expected fake exc:a random IOException (_r.fld)
>>    [junit4]   1> java.io.IOException: a random IOException (_r.fld)
>>    [junit4]   1>        at org.apache.lucene.store.MockDirectoryWrapper.maybeThrowIOExceptionOnOpen(MockDirectoryWrapper.java:456)
>>    [junit4]   1>        at org.apache.lucene.store.MockDirectoryWrapper.createOutput(MockDirectoryWrapper.java:548)
>>    [junit4]   1>        at org.apache.lucene.store.LockValidatingDirectoryWrapper.createOutput(LockValidatingDirectoryWrapper.java:44)
>>    [junit4]   1>        at org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:43)
>>    [junit4]   1>        at org.apache.lucene.codecs.simpletext.SimpleTextStoredFieldsWriter.<init>(SimpleTextStoredFieldsWriter.java:69)
>>    [junit4]   1>        at org.apache.lucene.codecs.simpletext.SimpleTextStoredFieldsFormat.fieldsWriter(SimpleTextStoredFieldsFormat.java:45)
>>    [junit4]   1>        at org.apache.lucene.index.DefaultIndexingChain.initStoredFieldsWriter(DefaultIndexingChain.java:81)
>>    [junit4]   1>        at org.apache.lucene.index.DefaultIndexingChain.startStoredFields(DefaultIndexingChain.java:258)
>>    [junit4]   1>        at org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:330)
>>    [junit4]   1>        at org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:234)
>>    [junit4]   1>        at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:457)
>>    [junit4]   1>        at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1458)
>>    [junit4]   1>        at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1237)
>>    [junit4]   1>        at org.apache.lucene.index.BaseIndexFileFormatTestCase.testRandomExceptions(BaseIndexFileFormatTestCase.java:427)
>>    [junit4]   1>        at org.apache.lucene.index.BaseFieldInfoFormatTestCase.testRandomExceptions(BaseFieldInfoFormatTestCase.java:50)
>>    [junit4]   1>        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>    [junit4]   1>        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>>    [junit4]   1>        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>    [junit4]   1>        at java.lang.reflect.Method.invoke(Method.java:483)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
>>    [junit4]   1>        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
>>    [junit4]   1>        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
>>    [junit4]   1>        at java.lang.Thread.run(Thread.java:745)
>>    [junit4]   1>
>>    [junit4]   1> TEST: got expected fake exc:a random IOException (_1a_1.liv)
>>    [junit4]   1> java.nio.file.NoSuchFileException: a random IOException (_1a_1.liv)
>>    [junit4]   1>        at org.apache.lucene.store.MockDirectoryWrapper.maybeThrowIOExceptionOnOpen(MockDirectoryWrapper.java:458)
>>    [junit4]   1>        at org.apache.lucene.store.MockDirectoryWrapper.createOutput(MockDirectoryWrapper.java:548)
>>    [junit4]   1>        at org.apache.lucene.store.LockValidatingDirectoryWrapper.createOutput(LockValidatingDirectoryWrapper.java:44)
>>    [junit4]   1>        at org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:43)
>>    [junit4]   1>        at org.apache.lucene.codecs.simpletext.SimpleTextLiveDocsFormat.writeLiveDocs(SimpleTextLiveDocsFormat.java:121)
>>    [junit4]   1>        at org.apache.lucene.index.DocumentsWriterPerThread.sealFlushedSegment(DocumentsWriterPerThread.java:528)
>>    [junit4]   1>        at org.apache.lucene.index.DocumentsWriterPerThread.flush(DocumentsWriterPerThread.java:460)
>>    [junit4]   1>        at org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:510)
>>    [junit4]   1>        at org.apache.lucene.index.DocumentsWriter.flushAllThreads(DocumentsWriter.java:622)
>>    [junit4]   1>        at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:422)
>>    [junit4]   1>        at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:86)
>>    [junit4]   1>        at org.apache.lucene.index.BaseIndexFileFormatTestCase.testRandomExceptions(BaseIndexFileFormatTestCase.java:457)
>>    [junit4]   1>        at org.apache.lucene.index.BaseFieldInfoFormatTestCase.testRandomExceptions(BaseFieldInfoFormatTestCase.java:50)
>>    [junit4]   1>        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>    [junit4]   1>        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>>    [junit4]   1>        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>    [junit4]   1>        at java.lang.reflect.Method.invoke(Method.java:483)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
>>    [junit4]   1>        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
>>    [junit4]   1>        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
>>    [junit4]   1>        at java.lang.Thread.run(Thread.java:745)
>>    [junit4]   1>
>>    [junit4]   1> TEST: got expected fake exc:a random IOException (_1d.pst)
>>    [junit4]   1> java.io.FileNotFoundException: a random IOException (_1d.pst)
>>    [junit4]   1>        at org.apache.lucene.store.MockDirectoryWrapper.maybeThrowIOExceptionOnOpen(MockDirectoryWrapper.java:458)
>>    [junit4]   1>        at org.apache.lucene.store.MockDirectoryWrapper.openInput(MockDirectoryWrapper.java:635)
>>    [junit4]   1>        at org.apache.lucene.store.FilterDirectory.openInput(FilterDirectory.java:89)
>>    [junit4]   1>        at org.apache.lucene.store.FilterDirectory.openInput(FilterDirectory.java:89)
>>    [junit4]   1>        at org.apache.lucene.store.FilterDirectory.openInput(FilterDirectory.java:89)
>>    [junit4]   1>        at org.apache.lucene.codecs.simpletext.SimpleTextCompoundFormat.write(SimpleTextCompoundFormat.java:179)
>>    [junit4]   1>        at org.apache.lucene.index.IndexWriter.createCompoundFile(IndexWriter.java:4652)
>>    [junit4]   1>        at org.apache.lucene.index.DocumentsWriterPerThread.sealFlushedSegment(DocumentsWriterPerThread.java:493)
>>    [junit4]   1>        at org.apache.lucene.index.DocumentsWriterPerThread.flush(DocumentsWriterPerThread.java:460)
>>    [junit4]   1>        at org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:510)
>>    [junit4]   1>        at org.apache.lucene.index.DocumentsWriter.flushAllThreads(DocumentsWriter.java:622)
>>    [junit4]   1>        at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:422)
>>    [junit4]   1>        at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:86)
>>    [junit4]   1>        at org.apache.lucene.index.BaseIndexFileFormatTestCase.testRandomExceptions(BaseIndexFileFormatTestCase.java:457)
>>    [junit4]   1>        at org.apache.lucene.index.BaseFieldInfoFormatTestCase.testRandomExceptions(BaseFieldInfoFormatTestCase.java:50)
>>    [junit4]   1>        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>    [junit4]   1>        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>>    [junit4]   1>        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>    [junit4]   1>        at java.lang.reflect.Method.invoke(Method.java:483)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
>>    [junit4]   1>        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
>>    [junit4]   1>        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
>>    [junit4]   1>        at java.lang.Thread.run(Thread.java:745)
>>    [junit4]   1>
>>    [junit4]   1> TEST: got expected fake exc:a random IOException (_1h.scf)
>>    [junit4]   1> java.io.IOException: a random IOException (_1h.scf)
>>    [junit4]   1>        at org.apache.lucene.store.MockDirectoryWrapper.maybeThrowIOExceptionOnOpen(MockDirectoryWrapper.java:456)
>>    [junit4]   1>        at org.apache.lucene.store.MockDirectoryWrapper.openInput(MockDirectoryWrapper.java:635)
>>    [junit4]   1>        at org.apache.lucene.codecs.simpletext.SimpleTextCompoundFormat.getCompoundReader(SimpleTextCompoundFormat.java:58)
>>    [junit4]   1>        at org.apache.lucene.index.SegmentCoreReaders.<init>(SegmentCoreReaders.java:93)
>>    [junit4]   1>        at org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:65)
>>    [junit4]   1>        at org.apache.lucene.index.ReadersAndUpdates.getReader(ReadersAndUpdates.java:145)
>>    [junit4]   1>        at org.apache.lucene.index.ReadersAndUpdates.getReadOnlyClone(ReadersAndUpdates.java:197)
>>    [junit4]   1>        at org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:99)
>>    [junit4]   1>        at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:433)
>>    [junit4]   1>        at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:86)
>>    [junit4]   1>        at org.apache.lucene.index.BaseIndexFileFormatTestCase.testRandomExceptions(BaseIndexFileFormatTestCase.java:457)
>>    [junit4]   1>        at org.apache.lucene.index.BaseFieldInfoFormatTestCase.testRandomExceptions(BaseFieldInfoFormatTestCase.java:50)
>>    [junit4]   1>        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>    [junit4]   1>        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>>    [junit4]   1>        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>    [junit4]   1>        at java.lang.reflect.Method.invoke(Method.java:483)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
>>    [junit4]   1>        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
>>    [junit4]   1>        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
>>    [junit4]   1>        at java.lang.Thread.run(Thread.java:745)
>>    [junit4]   1>
>>    [junit4]   1> TEST: got expected fake exc:a random IOException (_1q.fld)
>>    [junit4]   1> java.io.IOException: a random IOException (_1q.fld)
>>    [junit4]   1>        at org.apache.lucene.store.MockDirectoryWrapper.maybeThrowIOExceptionOnOpen(MockDirectoryWrapper.java:456)
>>    [junit4]   1>        at org.apache.lucene.store.MockDirectoryWrapper.openInput(MockDirectoryWrapper.java:635)
>>    [junit4]   1>        at org.apache.lucene.store.FilterDirectory.openInput(FilterDirectory.java:89)
>>    [junit4]   1>        at org.apache.lucene.store.FilterDirectory.openInput(FilterDirectory.java:89)
>>    [junit4]   1>        at org.apache.lucene.store.FilterDirectory.openInput(FilterDirectory.java:89)
>>    [junit4]   1>        at org.apache.lucene.codecs.simpletext.SimpleTextCompoundFormat.write(SimpleTextCompoundFormat.java:179)
>>    [junit4]   1>        at org.apache.lucene.index.IndexWriter.createCompoundFile(IndexWriter.java:4652)
>>    [junit4]   1>        at org.apache.lucene.index.DocumentsWriterPerThread.sealFlushedSegment(DocumentsWriterPerThread.java:493)
>>    [junit4]   1>        at org.apache.lucene.index.DocumentsWriterPerThread.flush(DocumentsWriterPerThread.java:460)
>>    [junit4]   1>        at org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:510)
>>    [junit4]   1>        at org.apache.lucene.index.DocumentsWriter.flushAllThreads(DocumentsWriter.java:622)
>>    [junit4]   1>        at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:422)
>>    [junit4]   1>        at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:86)
>>    [junit4]   1>        at org.apache.lucene.index.BaseIndexFileFormatTestCase.testRandomExceptions(BaseIndexFileFormatTestCase.java:457)
>>    [junit4]   1>        at org.apache.lucene.index.BaseFieldInfoFormatTestCase.testRandomExceptions(BaseFieldInfoFormatTestCase.java:50)
>>    [junit4]   1>        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>    [junit4]   1>        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>>    [junit4]   1>        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>    [junit4]   1>        at java.lang.reflect.Method.invoke(Method.java:483)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
>>    [junit4]   1>        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
>>    [junit4]   1>        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
>>    [junit4]   1>        at java.lang.Thread.run(Thread.java:745)
>>    [junit4]   1>
>>    [junit4]   1> TEST: got expected fake exc:a random IOException (_29.fld)
>>    [junit4]   1> java.io.IOException: a random IOException (_29.fld)
>>    [junit4]   1>        at org.apache.lucene.store.MockDirectoryWrapper.maybeThrowIOExceptionOnOpen(MockDirectoryWrapper.java:456)
>>    [junit4]   1>        at org.apache.lucene.store.MockDirectoryWrapper.createOutput(MockDirectoryWrapper.java:548)
>>    [junit4]   1>        at org.apache.lucene.store.LockValidatingDirectoryWrapper.createOutput(LockValidatingDirectoryWrapper.java:44)
>>    [junit4]   1>        at org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:43)
>>    [junit4]   1>        at org.apache.lucene.codecs.simpletext.SimpleTextStoredFieldsWriter.<init>(SimpleTextStoredFieldsWriter.java:69)
>>    [junit4]   1>        at org.apache.lucene.codecs.simpletext.SimpleTextStoredFieldsFormat.fieldsWriter(SimpleTextStoredFieldsFormat.java:45)
>>    [junit4]   1>        at org.apache.lucene.index.DefaultIndexingChain.initStoredFieldsWriter(DefaultIndexingChain.java:81)
>>    [junit4]   1>        at org.apache.lucene.index.DefaultIndexingChain.startStoredFields(DefaultIndexingChain.java:258)
>>    [junit4]   1>        at org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:330)
>>    [junit4]   1>        at org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:234)
>>    [junit4]   1>        at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:457)
>>    [junit4]   1>        at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1458)
>>    [junit4]   1>        at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1237)
>>    [junit4]   1>        at org.apache.lucene.index.BaseIndexFileFormatTestCase.testRandomExceptions(BaseIndexFileFormatTestCase.java:427)
>>    [junit4]   1>        at org.apache.lucene.index.BaseFieldInfoFormatTestCase.testRandomExceptions(BaseFieldInfoFormatTestCase.java:50)
>>    [junit4]   1>        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>    [junit4]   1>        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>>    [junit4]   1>        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>    [junit4]   1>        at java.lang.reflect.Method.invoke(Method.java:483)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
>>    [junit4]   1>        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
>>    [junit4]   1>        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
>>    [junit4]   1>        at java.lang.Thread.run(Thread.java:745)
>>    [junit4]   1>
>>    [junit4]   1> TEST: got expected fake exc:a random IOException (_29_1.liv)
>>    [junit4]   1> java.io.FileNotFoundException: a random IOException (_29_1.liv)
>>    [junit4]   1>        at org.apache.lucene.store.MockDirectoryWrapper.maybeThrowIOExceptionOnOpen(MockDirectoryWrapper.java:458)
>>    [junit4]   1>        at org.apache.lucene.store.MockDirectoryWrapper.openInput(MockDirectoryWrapper.java:635)
>>    [junit4]   1>        at org.apache.lucene.store.Directory.openChecksumInput(Directory.java:109)
>>    [junit4]   1>        at org.apache.lucene.store.MockDirectoryWrapper.openChecksumInput(MockDirectoryWrapper.java:1009)
>>    [junit4]   1>        at org.apache.lucene.codecs.simpletext.SimpleTextLiveDocsFormat.readLiveDocs(SimpleTextLiveDocsFormat.java:77)
>>    [junit4]   1>        at org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:73)
>>    [junit4]   1>        at org.apache.lucene.index.ReadersAndUpdates.getReader(ReadersAndUpdates.java:145)
>>    [junit4]   1>        at org.apache.lucene.index.ReadersAndUpdates.getReadOnlyClone(ReadersAndUpdates.java:197)
>>    [junit4]   1>        at org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:99)
>>    [junit4]   1>        at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:433)
>>    [junit4]   1>        at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:86)
>>    [junit4]   1>        at org.apache.lucene.index.BaseIndexFileFormatTestCase.testRandomExceptions(BaseIndexFileFormatTestCase.java:457)
>>    [junit4]   1>        at org.apache.lucene.index.BaseFieldInfoFormatTestCase.testRandomExceptions(BaseFieldInfoFormatTestCase.java:50)
>>    [junit4]   1>        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>    [junit4]   1>        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>>    [junit4]   1>        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>    [junit4]   1>        at java.lang.reflect.Method.invoke(Method.java:483)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
>>    [junit4]   1>        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
>>    [junit4]   1>        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
>>    [junit4]   1>        at java.lang.Thread.run(Thread.java:745)
>>    [junit4]   1>
>>    [junit4]   1> TEST: got expected fake exc:a random IOException (_2o.pst)
>>    [junit4]   1> java.io.IOException: a random IOException (_2o.pst)
>>    [junit4]   1>        at org.apache.lucene.store.MockDirectoryWrapper.maybeThrowIOExceptionOnOpen(MockDirectoryWrapper.java:456)
>>    [junit4]   1>        at org.apache.lucene.store.MockDirectoryWrapper.createOutput(MockDirectoryWrapper.java:548)
>>    [junit4]   1>        at org.apache.lucene.store.LockValidatingDirectoryWrapper.createOutput(LockValidatingDirectoryWrapper.java:44)
>>    [junit4]   1>        at org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:43)
>>    [junit4]   1>        at org.apache.lucene.codecs.simpletext.SimpleTextFieldsWriter.<init>(SimpleTextFieldsWriter.java:52)
>>    [junit4]   1>        at org.apache.lucene.codecs.simpletext.SimpleTextPostingsFormat.fieldsConsumer(SimpleTextPostingsFormat.java:45)
>>    [junit4]   1>        at org.apache.lucene.index.FreqProxTermsWriter.flush(FreqProxTermsWriter.java:104)
>>    [junit4]   1>        at org.apache.lucene.index.DefaultIndexingChain.flush(DefaultIndexingChain.java:112)
>>    [junit4]   1>        at org.apache.lucene.index.DocumentsWriterPerThread.flush(DocumentsWriterPerThread.java:423)
>>    [junit4]   1>        at org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:510)
>>    [junit4]   1>        at org.apache.lucene.index.DocumentsWriter.flushAllThreads(DocumentsWriter.java:622)
>>    [junit4]   1>        at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:422)
>>    [junit4]   1>        at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:86)
>>    [junit4]   1>        at org.apache.lucene.index.BaseIndexFileFormatTestCase.testRandomExceptions(BaseIndexFileFormatTestCase.java:457)
>>    [junit4]   1>        at org.apache.lucene.index.BaseFieldInfoFormatTestCase.testRandomExceptions(BaseFieldInfoFormatTestCase.java:50)
>>    [junit4]   1>        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>    [junit4]   1>        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>>    [junit4]   1>        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>    [junit4]   1>        at java.lang.reflect.Method.invoke(Method.java:483)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
>>    [junit4]   1>        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
>>    [junit4]   1>        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
>>    [junit4]   1>        at java.lang.Thread.run(Thread.java:745)
>>    [junit4]   1>
>>    [junit4]   1> TEST: got expected fake exc:a random IOException (_2z.pst)
>>    [junit4]   1> java.io.IOException: a random IOException (_2z.pst)
>>    [junit4]   1>        at org.apache.lucene.store.MockDirectoryWrapper.maybeThrowIOExceptionOnOpen(MockDirectoryWrapper.java:456)
>>    [junit4]   1>        at org.apache.lucene.store.MockDirectoryWrapper.openInput(MockDirectoryWrapper.java:635)
>>    [junit4]   1>        at org.apache.lucene.codecs.simpletext.SimpleTextFieldsReader.<init>(SimpleTextFieldsReader.java:84)
>>    [junit4]   1>        at org.apache.lucene.codecs.simpletext.SimpleTextPostingsFormat.fieldsProducer(SimpleTextPostingsFormat.java:50)
>>    [junit4]   1>        at org.apache.lucene.index.SegmentCoreReaders.<init>(SegmentCoreReaders.java:104)
>>    [junit4]   1>        at org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:65)
>>    [junit4]   1>        at org.apache.lucene.index.ReadersAndUpdates.getReader(ReadersAndUpdates.java:145)
>>    [junit4]   1>        at org.apache.lucene.index.ReadersAndUpdates.getReadOnlyClone(ReadersAndUpdates.java:197)
>>    [junit4]   1>        at org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:99)
>>    [junit4]   1>        at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:433)
>>    [junit4]   1>        at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:86)
>>    [junit4]   1>        at org.apache.lucene.index.BaseIndexFileFormatTestCase.testRandomExceptions(BaseIndexFileFormatTestCase.java:457)
>>    [junit4]   1>        at org.apache.lucene.index.BaseFieldInfoFormatTestCase.testRandomExceptions(BaseFieldInfoFormatTestCase.java:50)
>>    [junit4]   1>        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>    [junit4]   1>        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>>    [junit4]   1>        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>    [junit4]   1>        at java.lang.reflect.Method.invoke(Method.java:483)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
>>    [junit4]   1>        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
>>    [junit4]   1>        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
>>    [junit4]   1>        at java.lang.Thread.run(Thread.java:745)
>>    [junit4]   1>
>>    [junit4]   1> TEST: got expected fake exc:a random IOException (_33.fld)
>>    [junit4]   1> java.io.IOException: a random IOException (_33.fld)
>>    [junit4]   1>        at org.apache.lucene.store.MockDirectoryWrapper.maybeThrowIOExceptionOnOpen(MockDirectoryWrapper.java:456)
>>    [junit4]   1>        at org.apache.lucene.store.MockDirectoryWrapper.createOutput(MockDirectoryWrapper.java:548)
>>    [junit4]   1>        at org.apache.lucene.store.LockValidatingDirectoryWrapper.createOutput(LockValidatingDirectoryWrapper.java:44)
>>    [junit4]   1>        at org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:43)
>>    [junit4]   1>        at org.apache.lucene.codecs.simpletext.SimpleTextStoredFieldsWriter.<init>(SimpleTextStoredFieldsWriter.java:69)
>>    [junit4]   1>        at org.apache.lucene.codecs.simpletext.SimpleTextStoredFieldsFormat.fieldsWriter(SimpleTextStoredFieldsFormat.java:45)
>>    [junit4]   1>        at org.apache.lucene.index.DefaultIndexingChain.initStoredFieldsWriter(DefaultIndexingChain.java:81)
>>    [junit4]   1>        at org.apache.lucene.index.DefaultIndexingChain.startStoredFields(DefaultIndexingChain.java:258)
>>    [junit4]   1>        at org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:330)
>>    [junit4]   1>        at org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:234)
>>    [junit4]   1>        at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:457)
>>    [junit4]   1>        at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1458)
>>    [junit4]   1>        at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1237)
>>    [junit4]   1>        at org.apache.lucene.index.BaseIndexFileFormatTestCase.testRandomExceptions(BaseIndexFileFormatTestCase.java:427)
>>    [junit4]   1>        at org.apache.lucene.index.BaseFieldInfoFormatTestCase.testRandomExceptions(BaseFieldInfoFormatTestCase.java:50)
>>    [junit4]   1>        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>    [junit4]   1>        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>>    [junit4]   1>        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>    [junit4]   1>        at java.lang.reflect.Method.invoke(Method.java:483)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
>>    [junit4]   1>        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
>>    [junit4]   1>        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
>>    [junit4]   1>        at java.lang.Thread.run(Thread.java:745)
>>    [junit4]   1>
>>    [junit4]   1> TEST: got expected fake exc:a random IOException (_3e.pst)
>>    [junit4]   1> java.io.IOException: a random IOException (_3e.pst)
>>    [junit4]   1>        at org.apache.lucene.store.MockDirectoryWrapper.maybeThrowIOExceptionOnOpen(MockDirectoryWrapper.java:456)
>>    [junit4]   1>        at org.apache.lucene.store.MockDirectoryWrapper.createOutput(MockDirectoryWrapper.java:548)
>>    [junit4]   1>        at org.apache.lucene.store.LockValidatingDirectoryWrapper.createOutput(LockValidatingDirectoryWrapper.java:44)
>>    [junit4]   1>        at org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:43)
>>    [junit4]   1>        at org.apache.lucene.codecs.simpletext.SimpleTextFieldsWriter.<init>(SimpleTextFieldsWriter.java:52)
>>    [junit4]   1>        at org.apache.lucene.codecs.simpletext.SimpleTextPostingsFormat.fieldsConsumer(SimpleTextPostingsFormat.java:45)
>>    [junit4]   1>        at org.apache.lucene.index.FreqProxTermsWriter.flush(FreqProxTermsWriter.java:104)
>>    [junit4]   1>        at org.apache.lucene.index.DefaultIndexingChain.flush(DefaultIndexingChain.java:112)
>>    [junit4]   1>        at org.apache.lucene.index.DocumentsWriterPerThread.flush(DocumentsWriterPerThread.java:423)
>>    [junit4]   1>        at org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:510)
>>    [junit4]   1>        at org.apache.lucene.index.DocumentsWriter.flushAllThreads(DocumentsWriter.java:622)
>>    [junit4]   1>        at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:422)
>>    [junit4]   1>        at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:86)
>>    [junit4]   1>        at org.apache
>>
>> [...truncated too long message...]
>>
>> LockValidatingDirectoryWrapper.java:44)
>>    [junit4]   1>        at org.apache.lucene.codecs.simpletext.SimpleTextSegmentInfoFormat.write(SimpleTextSegmentInfoFormat.java:159)
>>    [junit4]   1>        at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4187)
>>    [junit4]   1>        at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3639)
>>    [junit4]   1>        at org.apache.lucene.index.SerialMergeScheduler.merge(SerialMergeScheduler.java:40)
>>    [junit4]   1>        at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:1912)
>>    [junit4]   1>        at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:453)
>>    [junit4]   1>        at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:86)
>>    [junit4]   1>        at org.apache.lucene.index.BaseIndexFileFormatTestCase.testRandomExceptions(BaseIndexFileFormatTestCase.java:457)
>>    [junit4]   1>        at org.apache.lucene.index.BaseFieldInfoFormatTestCase.testRandomExceptions(BaseFieldInfoFormatTestCase.java:50)
>>    [junit4]   1>        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>    [junit4]   1>        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>>    [junit4]   1>        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>    [junit4]   1>        at java.lang.reflect.Method.invoke(Method.java:483)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
>>    [junit4]   1>        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
>>    [junit4]   1>        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
>>    [junit4]   1>        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
>>    [junit4]   1>        at java.lang.Thread.run(Thread.java:745)
>>    [junit4]   1>
>>    [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
>>    [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestSimpleTextFieldInfoFormat -Dtests.method=testRandomExceptions -Dtests.seed=A1871D12B75F5C9E -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=sq_AL -Dtests.timezone=CAT -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
>>    [junit4] FAILURE 3.05s J1 | TestSimpleTextFieldInfoFormat.testRandomExceptions <<<
>>    [junit4]    > Throwable #1: java.lang.AssertionError: hit unexpected FileNotFoundException: file=_3w.si
>>    [junit4]    >        at __randomizedtesting.SeedInfo.seed([A1871D12B75F5C9E:C9A873D62951053E]:0)
>>    [junit4]    >        at org.apache.lucene.index.IndexFileDeleter.deleteFile(IndexFileDeleter.java:751)
>>    [junit4]    >        at org.apache.lucene.index.IndexFileDeleter.deletePendingFiles(IndexFileDeleter.java:528)
>>    [junit4]    >        at org.apache.lucene.index.IndexFileDeleter.decRef(IndexFileDeleter.java:636)
>>    [junit4]    >        at org.apache.lucene.index.IndexFileDeleter.checkpoint(IndexFileDeleter.java:578)
>>    [junit4]    >        at org.apache.lucene.index.IndexWriter.checkpoint(IndexWriter.java:2298)
>>    [junit4]    >        at org.apache.lucene.index.IndexWriter.publishFlushedSegment(IndexWriter.java:2356)
>>    [junit4]    >        at org.apache.lucene.index.DocumentsWriterFlushQueue$FlushTicket.publishFlushedSegment(DocumentsWriterFlushQueue.java:198)
>>    [junit4]    >        at org.apache.lucene.index.DocumentsWriterFlushQueue$FlushTicket.finishFlush(DocumentsWriterFlushQueue.java:213)
>>    [junit4]    >        at org.apache.lucene.index.DocumentsWriterFlushQueue$SegmentFlushTicket.publish(DocumentsWriterFlushQueue.java:249)
>>    [junit4]    >        at org.apache.lucene.index.DocumentsWriterFlushQueue.innerPurge(DocumentsWriterFlushQueue.java:116)
>>    [junit4]    >        at org.apache.lucene.index.DocumentsWriterFlushQueue.forcePurge(DocumentsWriterFlushQueue.java:138)
>>    [junit4]    >        at org.apache.lucene.index.DocumentsWriter.flushAllThreads(DocumentsWriter.java:632)
>>    [junit4]    >        at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:422)
>>    [junit4]    >        at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:86)
>>    [junit4]    >        at org.apache.lucene.index.BaseIndexFileFormatTestCase.testRandomExceptions(BaseIndexFileFormatTestCase.java:457)
>>    [junit4]    >        at org.apache.lucene.index.BaseFieldInfoFormatTestCase.testRandomExceptions(BaseFieldInfoFormatTestCase.java:50)
>>    [junit4]    >        at java.lang.Thread.run(Thread.java:745)
>>    [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/build/codecs/test/J1/temp/lucene.codecs.simpletext.TestSimpleTextFieldInfoFormat_A1871D12B75F5C9E-001
>>    [junit4]   2> NOTE: test params are: codec=Asserting(Lucene53): {}, docValues:{}, sim=DefaultSimilarity, locale=sq_AL, timezone=CAT
>>    [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.8.0_25 (64-bit)/cpus=4,threads=1,free=216720192,total=253231104
>>    [junit4]   2> NOTE: All tests run in this JVM: [TestSimpleTextFieldInfoFormat]
>>    [junit4] Completed [1/21] on J1 in 5.76s, 11 tests, 1 failure <<< FAILURES!
>>
>> [...truncated 90 lines...]
>> BUILD FAILED
>> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:533: The following error occurred while executing this line:
>> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:474: The following error occurred while executing this line:
>> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:61: The following error occurred while executing this line:
>> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/extra-targets.xml:39: The following error occurred while executing this line:
>> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/build.xml:469: The following error occurred while executing this line:
>> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/common-build.xml:2156: The following error occurred while executing this line:
>> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/module-build.xml:58: The following error occurred while executing this line:
>> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/common-build.xml:1426: The following error occurred while executing this line:
>> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/common-build.xml:981: There were test failures: 21 suites, 616 tests, 1 failure, 1 ignored (1 assumption)
>>
>> Total time: 95 minutes 31 seconds
>> Build step 'Invoke Ant' marked build as failure
>> Archiving artifacts
>> No prior successful build to compare, so performing full copy of artifacts
>> Recording test results
>> Email was triggered for: Failure
>> Sending email for trigger: Failure
>>
>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: dev-help@lucene.apache.org

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Re: [JENKINS] Lucene-Solr-NightlyTests-trunk - Build # 739 - Still Failing

Posted by Michael McCandless <lu...@mikemccandless.com>.
I'll dig, this is a new assertion I added recently to IFD.

Mike McCandless

http://blog.mikemccandless.com


On Tue, Jul 14, 2015 at 12:04 AM, Apache Jenkins Server
<je...@builds.apache.org> wrote:
> Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-trunk/739/
>
> 1 tests failed.
> REGRESSION:  org.apache.lucene.codecs.simpletext.TestSimpleTextFieldInfoFormat.testRandomExceptions
>
> Error Message:
> hit unexpected FileNotFoundException: file=_3w.si
>
> Stack Trace:
> java.lang.AssertionError: hit unexpected FileNotFoundException: file=_3w.si
>         at __randomizedtesting.SeedInfo.seed([A1871D12B75F5C9E:C9A873D62951053E]:0)
>         at org.apache.lucene.index.IndexFileDeleter.deleteFile(IndexFileDeleter.java:751)
>         at org.apache.lucene.index.IndexFileDeleter.deletePendingFiles(IndexFileDeleter.java:528)
>         at org.apache.lucene.index.IndexFileDeleter.decRef(IndexFileDeleter.java:636)
>         at org.apache.lucene.index.IndexFileDeleter.checkpoint(IndexFileDeleter.java:578)
>         at org.apache.lucene.index.IndexWriter.checkpoint(IndexWriter.java:2298)
>         at org.apache.lucene.index.IndexWriter.publishFlushedSegment(IndexWriter.java:2356)
>         at org.apache.lucene.index.DocumentsWriterFlushQueue$FlushTicket.publishFlushedSegment(DocumentsWriterFlushQueue.java:198)
>         at org.apache.lucene.index.DocumentsWriterFlushQueue$FlushTicket.finishFlush(DocumentsWriterFlushQueue.java:213)
>         at org.apache.lucene.index.DocumentsWriterFlushQueue$SegmentFlushTicket.publish(DocumentsWriterFlushQueue.java:249)
>         at org.apache.lucene.index.DocumentsWriterFlushQueue.innerPurge(DocumentsWriterFlushQueue.java:116)
>         at org.apache.lucene.index.DocumentsWriterFlushQueue.forcePurge(DocumentsWriterFlushQueue.java:138)
>         at org.apache.lucene.index.DocumentsWriter.flushAllThreads(DocumentsWriter.java:632)
>         at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:422)
>         at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:86)
>         at org.apache.lucene.index.BaseIndexFileFormatTestCase.testRandomExceptions(BaseIndexFileFormatTestCase.java:457)
>         at org.apache.lucene.index.BaseFieldInfoFormatTestCase.testRandomExceptions(BaseFieldInfoFormatTestCase.java:50)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>         at java.lang.reflect.Method.invoke(Method.java:483)
>         at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
>         at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
>         at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
>         at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
>         at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
>         at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
>         at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
>         at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
>         at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
>         at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
>         at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
>         at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
>         at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
>         at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
>         at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>         at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
>         at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
>         at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>         at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
>         at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
>         at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
>         at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
>         at java.lang.Thread.run(Thread.java:745)
>
>
>
>
> Build Log:
> [...truncated 6446 lines...]
>    [junit4] Suite: org.apache.lucene.codecs.simpletext.TestSimpleTextFieldInfoFormat
>    [junit4]   1> Unexpected exception: dumping fake-exception-log:...
>    [junit4]   1>
>    [junit4]   1> TEST: got expected fake exc:a random IOException (_7.si)
>    [junit4]   1> java.io.IOException: a random IOException (_7.si)
>    [junit4]   1>        at org.apache.lucene.store.MockDirectoryWrapper.maybeThrowIOExceptionOnOpen(MockDirectoryWrapper.java:456)
>    [junit4]   1>        at org.apache.lucene.store.MockDirectoryWrapper.createOutput(MockDirectoryWrapper.java:548)
>    [junit4]   1>        at org.apache.lucene.store.LockValidatingDirectoryWrapper.createOutput(LockValidatingDirectoryWrapper.java:44)
>    [junit4]   1>        at org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:43)
>    [junit4]   1>        at org.apache.lucene.codecs.simpletext.SimpleTextSegmentInfoFormat.write(SimpleTextSegmentInfoFormat.java:159)
>    [junit4]   1>        at org.apache.lucene.index.DocumentsWriterPerThread.sealFlushedSegment(DocumentsWriterPerThread.java:502)
>    [junit4]   1>        at org.apache.lucene.index.DocumentsWriterPerThread.flush(DocumentsWriterPerThread.java:460)
>    [junit4]   1>        at org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:510)
>    [junit4]   1>        at org.apache.lucene.index.DocumentsWriter.flushAllThreads(DocumentsWriter.java:622)
>    [junit4]   1>        at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:422)
>    [junit4]   1>        at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:86)
>    [junit4]   1>        at org.apache.lucene.index.BaseIndexFileFormatTestCase.testRandomExceptions(BaseIndexFileFormatTestCase.java:457)
>    [junit4]   1>        at org.apache.lucene.index.BaseFieldInfoFormatTestCase.testRandomExceptions(BaseFieldInfoFormatTestCase.java:50)
>    [junit4]   1>        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>    [junit4]   1>        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>    [junit4]   1>        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>    [junit4]   1>        at java.lang.reflect.Method.invoke(Method.java:483)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
>    [junit4]   1>        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
>    [junit4]   1>        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
>    [junit4]   1>        at java.lang.Thread.run(Thread.java:745)
>    [junit4]   1>
>    [junit4]   1> TEST: got expected fake exc:a random IOException (_a.pst)
>    [junit4]   1> java.io.IOException: a random IOException (_a.pst)
>    [junit4]   1>        at org.apache.lucene.store.MockDirectoryWrapper.maybeThrowIOExceptionOnOpen(MockDirectoryWrapper.java:456)
>    [junit4]   1>        at org.apache.lucene.store.MockDirectoryWrapper.openInput(MockDirectoryWrapper.java:635)
>    [junit4]   1>        at org.apache.lucene.store.FilterDirectory.openInput(FilterDirectory.java:89)
>    [junit4]   1>        at org.apache.lucene.store.FilterDirectory.openInput(FilterDirectory.java:89)
>    [junit4]   1>        at org.apache.lucene.store.FilterDirectory.openInput(FilterDirectory.java:89)
>    [junit4]   1>        at org.apache.lucene.codecs.simpletext.SimpleTextCompoundFormat.write(SimpleTextCompoundFormat.java:179)
>    [junit4]   1>        at org.apache.lucene.index.IndexWriter.createCompoundFile(IndexWriter.java:4652)
>    [junit4]   1>        at org.apache.lucene.index.DocumentsWriterPerThread.sealFlushedSegment(DocumentsWriterPerThread.java:493)
>    [junit4]   1>        at org.apache.lucene.index.DocumentsWriterPerThread.flush(DocumentsWriterPerThread.java:460)
>    [junit4]   1>        at org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:510)
>    [junit4]   1>        at org.apache.lucene.index.DocumentsWriter.flushAllThreads(DocumentsWriter.java:622)
>    [junit4]   1>        at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:422)
>    [junit4]   1>        at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:86)
>    [junit4]   1>        at org.apache.lucene.index.BaseIndexFileFormatTestCase.testRandomExceptions(BaseIndexFileFormatTestCase.java:457)
>    [junit4]   1>        at org.apache.lucene.index.BaseFieldInfoFormatTestCase.testRandomExceptions(BaseFieldInfoFormatTestCase.java:50)
>    [junit4]   1>        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>    [junit4]   1>        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>    [junit4]   1>        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>    [junit4]   1>        at java.lang.reflect.Method.invoke(Method.java:483)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
>    [junit4]   1>        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
>    [junit4]   1>        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
>    [junit4]   1>        at java.lang.Thread.run(Thread.java:745)
>    [junit4]   1>
>    [junit4]   1> TEST: got expected fake exc:a random IOException (_e.fld)
>    [junit4]   1> java.io.FileNotFoundException: a random IOException (_e.fld)
>    [junit4]   1>        at org.apache.lucene.store.MockDirectoryWrapper.maybeThrowIOExceptionOnOpen(MockDirectoryWrapper.java:458)
>    [junit4]   1>        at org.apache.lucene.store.MockDirectoryWrapper.openInput(MockDirectoryWrapper.java:635)
>    [junit4]   1>        at org.apache.lucene.codecs.simpletext.SimpleTextStoredFieldsReader.<init>(SimpleTextStoredFieldsReader.java:68)
>    [junit4]   1>        at org.apache.lucene.codecs.simpletext.SimpleTextStoredFieldsFormat.fieldsReader(SimpleTextStoredFieldsFormat.java:40)
>    [junit4]   1>        at org.apache.lucene.index.SegmentCoreReaders.<init>(SegmentCoreReaders.java:117)
>    [junit4]   1>        at org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:65)
>    [junit4]   1>        at org.apache.lucene.index.ReadersAndUpdates.getReader(ReadersAndUpdates.java:145)
>    [junit4]   1>        at org.apache.lucene.index.BufferedUpdatesStream$SegmentState.<init>(BufferedUpdatesStream.java:378)
>    [junit4]   1>        at org.apache.lucene.index.BufferedUpdatesStream.openSegmentStates(BufferedUpdatesStream.java:410)
>    [junit4]   1>        at org.apache.lucene.index.BufferedUpdatesStream.applyDeletesAndUpdates(BufferedUpdatesStream.java:255)
>    [junit4]   1>        at org.apache.lucene.index.IndexWriter._mergeInit(IndexWriter.java:3820)
>    [junit4]   1>        at org.apache.lucene.index.IndexWriter.mergeInit(IndexWriter.java:3778)
>    [junit4]   1>        at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3630)
>    [junit4]   1>        at org.apache.lucene.index.SerialMergeScheduler.merge(SerialMergeScheduler.java:40)
>    [junit4]   1>        at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:1912)
>    [junit4]   1>        at org.apache.lucene.index.IndexWriter.doAfterSegmentFlushed(IndexWriter.java:4703)
>    [junit4]   1>        at org.apache.lucene.index.DocumentsWriter$MergePendingEvent.process(DocumentsWriter.java:697)
>    [junit4]   1>        at org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:4729)
>    [junit4]   1>        at org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:4720)
>    [junit4]   1>        at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1459)
>    [junit4]   1>        at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1237)
>    [junit4]   1>        at org.apache.lucene.index.BaseIndexFileFormatTestCase.testRandomExceptions(BaseIndexFileFormatTestCase.java:427)
>    [junit4]   1>        at org.apache.lucene.index.BaseFieldInfoFormatTestCase.testRandomExceptions(BaseFieldInfoFormatTestCase.java:50)
>    [junit4]   1>        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>    [junit4]   1>        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>    [junit4]   1>        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>    [junit4]   1>        at java.lang.reflect.Method.invoke(Method.java:483)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
>    [junit4]   1>        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
>    [junit4]   1>        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
>    [junit4]   1>        at java.lang.Thread.run(Thread.java:745)
>    [junit4]   1>
>    [junit4]   1> TEST: got expected fake exc:a random IOException (_n.inf)
>    [junit4]   1> java.io.IOException: a random IOException (_n.inf)
>    [junit4]   1>        at org.apache.lucene.store.MockDirectoryWrapper.maybeThrowIOExceptionOnOpen(MockDirectoryWrapper.java:456)
>    [junit4]   1>        at org.apache.lucene.store.MockDirectoryWrapper.openInput(MockDirectoryWrapper.java:635)
>    [junit4]   1>        at org.apache.lucene.store.Directory.openChecksumInput(Directory.java:109)
>    [junit4]   1>        at org.apache.lucene.store.MockDirectoryWrapper.openChecksumInput(MockDirectoryWrapper.java:1009)
>    [junit4]   1>        at org.apache.lucene.codecs.simpletext.SimpleTextFieldInfosFormat.read(SimpleTextFieldInfosFormat.java:71)
>    [junit4]   1>        at org.apache.lucene.index.SegmentCoreReaders.<init>(SegmentCoreReaders.java:99)
>    [junit4]   1>        at org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:65)
>    [junit4]   1>        at org.apache.lucene.index.ReadersAndUpdates.getReader(ReadersAndUpdates.java:145)
>    [junit4]   1>        at org.apache.lucene.index.BufferedUpdatesStream$SegmentState.<init>(BufferedUpdatesStream.java:378)
>    [junit4]   1>        at org.apache.lucene.index.BufferedUpdatesStream.openSegmentStates(BufferedUpdatesStream.java:410)
>    [junit4]   1>        at org.apache.lucene.index.BufferedUpdatesStream.applyDeletesAndUpdates(BufferedUpdatesStream.java:255)
>    [junit4]   1>        at org.apache.lucene.index.IndexWriter._mergeInit(IndexWriter.java:3820)
>    [junit4]   1>        at org.apache.lucene.index.IndexWriter.mergeInit(IndexWriter.java:3778)
>    [junit4]   1>        at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3630)
>    [junit4]   1>        at org.apache.lucene.index.SerialMergeScheduler.merge(SerialMergeScheduler.java:40)
>    [junit4]   1>        at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:1912)
>    [junit4]   1>        at org.apache.lucene.index.IndexWriter.doAfterSegmentFlushed(IndexWriter.java:4703)
>    [junit4]   1>        at org.apache.lucene.index.DocumentsWriter$MergePendingEvent.process(DocumentsWriter.java:697)
>    [junit4]   1>        at org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:4729)
>    [junit4]   1>        at org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:4720)
>    [junit4]   1>        at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1459)
>    [junit4]   1>        at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1237)
>    [junit4]   1>        at org.apache.lucene.index.BaseIndexFileFormatTestCase.testRandomExceptions(BaseIndexFileFormatTestCase.java:427)
>    [junit4]   1>        at org.apache.lucene.index.BaseFieldInfoFormatTestCase.testRandomExceptions(BaseFieldInfoFormatTestCase.java:50)
>    [junit4]   1>        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>    [junit4]   1>        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>    [junit4]   1>        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>    [junit4]   1>        at java.lang.reflect.Method.invoke(Method.java:483)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
>    [junit4]   1>        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
>    [junit4]   1>        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
>    [junit4]   1>        at java.lang.Thread.run(Thread.java:745)
>    [junit4]   1>
>    [junit4]   1> TEST: got expected fake exc:a random IOException (_r.fld)
>    [junit4]   1> java.io.IOException: a random IOException (_r.fld)
>    [junit4]   1>        at org.apache.lucene.store.MockDirectoryWrapper.maybeThrowIOExceptionOnOpen(MockDirectoryWrapper.java:456)
>    [junit4]   1>        at org.apache.lucene.store.MockDirectoryWrapper.createOutput(MockDirectoryWrapper.java:548)
>    [junit4]   1>        at org.apache.lucene.store.LockValidatingDirectoryWrapper.createOutput(LockValidatingDirectoryWrapper.java:44)
>    [junit4]   1>        at org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:43)
>    [junit4]   1>        at org.apache.lucene.codecs.simpletext.SimpleTextStoredFieldsWriter.<init>(SimpleTextStoredFieldsWriter.java:69)
>    [junit4]   1>        at org.apache.lucene.codecs.simpletext.SimpleTextStoredFieldsFormat.fieldsWriter(SimpleTextStoredFieldsFormat.java:45)
>    [junit4]   1>        at org.apache.lucene.index.DefaultIndexingChain.initStoredFieldsWriter(DefaultIndexingChain.java:81)
>    [junit4]   1>        at org.apache.lucene.index.DefaultIndexingChain.startStoredFields(DefaultIndexingChain.java:258)
>    [junit4]   1>        at org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:330)
>    [junit4]   1>        at org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:234)
>    [junit4]   1>        at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:457)
>    [junit4]   1>        at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1458)
>    [junit4]   1>        at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1237)
>    [junit4]   1>        at org.apache.lucene.index.BaseIndexFileFormatTestCase.testRandomExceptions(BaseIndexFileFormatTestCase.java:427)
>    [junit4]   1>        at org.apache.lucene.index.BaseFieldInfoFormatTestCase.testRandomExceptions(BaseFieldInfoFormatTestCase.java:50)
>    [junit4]   1>        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>    [junit4]   1>        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>    [junit4]   1>        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>    [junit4]   1>        at java.lang.reflect.Method.invoke(Method.java:483)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
>    [junit4]   1>        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
>    [junit4]   1>        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
>    [junit4]   1>        at java.lang.Thread.run(Thread.java:745)
>    [junit4]   1>
>    [junit4]   1> TEST: got expected fake exc:a random IOException (_1a_1.liv)
>    [junit4]   1> java.nio.file.NoSuchFileException: a random IOException (_1a_1.liv)
>    [junit4]   1>        at org.apache.lucene.store.MockDirectoryWrapper.maybeThrowIOExceptionOnOpen(MockDirectoryWrapper.java:458)
>    [junit4]   1>        at org.apache.lucene.store.MockDirectoryWrapper.createOutput(MockDirectoryWrapper.java:548)
>    [junit4]   1>        at org.apache.lucene.store.LockValidatingDirectoryWrapper.createOutput(LockValidatingDirectoryWrapper.java:44)
>    [junit4]   1>        at org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:43)
>    [junit4]   1>        at org.apache.lucene.codecs.simpletext.SimpleTextLiveDocsFormat.writeLiveDocs(SimpleTextLiveDocsFormat.java:121)
>    [junit4]   1>        at org.apache.lucene.index.DocumentsWriterPerThread.sealFlushedSegment(DocumentsWriterPerThread.java:528)
>    [junit4]   1>        at org.apache.lucene.index.DocumentsWriterPerThread.flush(DocumentsWriterPerThread.java:460)
>    [junit4]   1>        at org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:510)
>    [junit4]   1>        at org.apache.lucene.index.DocumentsWriter.flushAllThreads(DocumentsWriter.java:622)
>    [junit4]   1>        at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:422)
>    [junit4]   1>        at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:86)
>    [junit4]   1>        at org.apache.lucene.index.BaseIndexFileFormatTestCase.testRandomExceptions(BaseIndexFileFormatTestCase.java:457)
>    [junit4]   1>        at org.apache.lucene.index.BaseFieldInfoFormatTestCase.testRandomExceptions(BaseFieldInfoFormatTestCase.java:50)
>    [junit4]   1>        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>    [junit4]   1>        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>    [junit4]   1>        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>    [junit4]   1>        at java.lang.reflect.Method.invoke(Method.java:483)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
>    [junit4]   1>        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
>    [junit4]   1>        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
>    [junit4]   1>        at java.lang.Thread.run(Thread.java:745)
>    [junit4]   1>
>    [junit4]   1> TEST: got expected fake exc:a random IOException (_1d.pst)
>    [junit4]   1> java.io.FileNotFoundException: a random IOException (_1d.pst)
>    [junit4]   1>        at org.apache.lucene.store.MockDirectoryWrapper.maybeThrowIOExceptionOnOpen(MockDirectoryWrapper.java:458)
>    [junit4]   1>        at org.apache.lucene.store.MockDirectoryWrapper.openInput(MockDirectoryWrapper.java:635)
>    [junit4]   1>        at org.apache.lucene.store.FilterDirectory.openInput(FilterDirectory.java:89)
>    [junit4]   1>        at org.apache.lucene.store.FilterDirectory.openInput(FilterDirectory.java:89)
>    [junit4]   1>        at org.apache.lucene.store.FilterDirectory.openInput(FilterDirectory.java:89)
>    [junit4]   1>        at org.apache.lucene.codecs.simpletext.SimpleTextCompoundFormat.write(SimpleTextCompoundFormat.java:179)
>    [junit4]   1>        at org.apache.lucene.index.IndexWriter.createCompoundFile(IndexWriter.java:4652)
>    [junit4]   1>        at org.apache.lucene.index.DocumentsWriterPerThread.sealFlushedSegment(DocumentsWriterPerThread.java:493)
>    [junit4]   1>        at org.apache.lucene.index.DocumentsWriterPerThread.flush(DocumentsWriterPerThread.java:460)
>    [junit4]   1>        at org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:510)
>    [junit4]   1>        at org.apache.lucene.index.DocumentsWriter.flushAllThreads(DocumentsWriter.java:622)
>    [junit4]   1>        at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:422)
>    [junit4]   1>        at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:86)
>    [junit4]   1>        at org.apache.lucene.index.BaseIndexFileFormatTestCase.testRandomExceptions(BaseIndexFileFormatTestCase.java:457)
>    [junit4]   1>        at org.apache.lucene.index.BaseFieldInfoFormatTestCase.testRandomExceptions(BaseFieldInfoFormatTestCase.java:50)
>    [junit4]   1>        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>    [junit4]   1>        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>    [junit4]   1>        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>    [junit4]   1>        at java.lang.reflect.Method.invoke(Method.java:483)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
>    [junit4]   1>        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
>    [junit4]   1>        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
>    [junit4]   1>        at java.lang.Thread.run(Thread.java:745)
>    [junit4]   1>
>    [junit4]   1> TEST: got expected fake exc:a random IOException (_1h.scf)
>    [junit4]   1> java.io.IOException: a random IOException (_1h.scf)
>    [junit4]   1>        at org.apache.lucene.store.MockDirectoryWrapper.maybeThrowIOExceptionOnOpen(MockDirectoryWrapper.java:456)
>    [junit4]   1>        at org.apache.lucene.store.MockDirectoryWrapper.openInput(MockDirectoryWrapper.java:635)
>    [junit4]   1>        at org.apache.lucene.codecs.simpletext.SimpleTextCompoundFormat.getCompoundReader(SimpleTextCompoundFormat.java:58)
>    [junit4]   1>        at org.apache.lucene.index.SegmentCoreReaders.<init>(SegmentCoreReaders.java:93)
>    [junit4]   1>        at org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:65)
>    [junit4]   1>        at org.apache.lucene.index.ReadersAndUpdates.getReader(ReadersAndUpdates.java:145)
>    [junit4]   1>        at org.apache.lucene.index.ReadersAndUpdates.getReadOnlyClone(ReadersAndUpdates.java:197)
>    [junit4]   1>        at org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:99)
>    [junit4]   1>        at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:433)
>    [junit4]   1>        at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:86)
>    [junit4]   1>        at org.apache.lucene.index.BaseIndexFileFormatTestCase.testRandomExceptions(BaseIndexFileFormatTestCase.java:457)
>    [junit4]   1>        at org.apache.lucene.index.BaseFieldInfoFormatTestCase.testRandomExceptions(BaseFieldInfoFormatTestCase.java:50)
>    [junit4]   1>        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>    [junit4]   1>        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>    [junit4]   1>        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>    [junit4]   1>        at java.lang.reflect.Method.invoke(Method.java:483)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
>    [junit4]   1>        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
>    [junit4]   1>        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
>    [junit4]   1>        at java.lang.Thread.run(Thread.java:745)
>    [junit4]   1>
>    [junit4]   1> TEST: got expected fake exc:a random IOException (_1q.fld)
>    [junit4]   1> java.io.IOException: a random IOException (_1q.fld)
>    [junit4]   1>        at org.apache.lucene.store.MockDirectoryWrapper.maybeThrowIOExceptionOnOpen(MockDirectoryWrapper.java:456)
>    [junit4]   1>        at org.apache.lucene.store.MockDirectoryWrapper.openInput(MockDirectoryWrapper.java:635)
>    [junit4]   1>        at org.apache.lucene.store.FilterDirectory.openInput(FilterDirectory.java:89)
>    [junit4]   1>        at org.apache.lucene.store.FilterDirectory.openInput(FilterDirectory.java:89)
>    [junit4]   1>        at org.apache.lucene.store.FilterDirectory.openInput(FilterDirectory.java:89)
>    [junit4]   1>        at org.apache.lucene.codecs.simpletext.SimpleTextCompoundFormat.write(SimpleTextCompoundFormat.java:179)
>    [junit4]   1>        at org.apache.lucene.index.IndexWriter.createCompoundFile(IndexWriter.java:4652)
>    [junit4]   1>        at org.apache.lucene.index.DocumentsWriterPerThread.sealFlushedSegment(DocumentsWriterPerThread.java:493)
>    [junit4]   1>        at org.apache.lucene.index.DocumentsWriterPerThread.flush(DocumentsWriterPerThread.java:460)
>    [junit4]   1>        at org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:510)
>    [junit4]   1>        at org.apache.lucene.index.DocumentsWriter.flushAllThreads(DocumentsWriter.java:622)
>    [junit4]   1>        at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:422)
>    [junit4]   1>        at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:86)
>    [junit4]   1>        at org.apache.lucene.index.BaseIndexFileFormatTestCase.testRandomExceptions(BaseIndexFileFormatTestCase.java:457)
>    [junit4]   1>        at org.apache.lucene.index.BaseFieldInfoFormatTestCase.testRandomExceptions(BaseFieldInfoFormatTestCase.java:50)
>    [junit4]   1>        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>    [junit4]   1>        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>    [junit4]   1>        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>    [junit4]   1>        at java.lang.reflect.Method.invoke(Method.java:483)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
>    [junit4]   1>        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
>    [junit4]   1>        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
>    [junit4]   1>        at java.lang.Thread.run(Thread.java:745)
>    [junit4]   1>
>    [junit4]   1> TEST: got expected fake exc:a random IOException (_29.fld)
>    [junit4]   1> java.io.IOException: a random IOException (_29.fld)
>    [junit4]   1>        at org.apache.lucene.store.MockDirectoryWrapper.maybeThrowIOExceptionOnOpen(MockDirectoryWrapper.java:456)
>    [junit4]   1>        at org.apache.lucene.store.MockDirectoryWrapper.createOutput(MockDirectoryWrapper.java:548)
>    [junit4]   1>        at org.apache.lucene.store.LockValidatingDirectoryWrapper.createOutput(LockValidatingDirectoryWrapper.java:44)
>    [junit4]   1>        at org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:43)
>    [junit4]   1>        at org.apache.lucene.codecs.simpletext.SimpleTextStoredFieldsWriter.<init>(SimpleTextStoredFieldsWriter.java:69)
>    [junit4]   1>        at org.apache.lucene.codecs.simpletext.SimpleTextStoredFieldsFormat.fieldsWriter(SimpleTextStoredFieldsFormat.java:45)
>    [junit4]   1>        at org.apache.lucene.index.DefaultIndexingChain.initStoredFieldsWriter(DefaultIndexingChain.java:81)
>    [junit4]   1>        at org.apache.lucene.index.DefaultIndexingChain.startStoredFields(DefaultIndexingChain.java:258)
>    [junit4]   1>        at org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:330)
>    [junit4]   1>        at org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:234)
>    [junit4]   1>        at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:457)
>    [junit4]   1>        at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1458)
>    [junit4]   1>        at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1237)
>    [junit4]   1>        at org.apache.lucene.index.BaseIndexFileFormatTestCase.testRandomExceptions(BaseIndexFileFormatTestCase.java:427)
>    [junit4]   1>        at org.apache.lucene.index.BaseFieldInfoFormatTestCase.testRandomExceptions(BaseFieldInfoFormatTestCase.java:50)
>    [junit4]   1>        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>    [junit4]   1>        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>    [junit4]   1>        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>    [junit4]   1>        at java.lang.reflect.Method.invoke(Method.java:483)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
>    [junit4]   1>        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
>    [junit4]   1>        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
>    [junit4]   1>        at java.lang.Thread.run(Thread.java:745)
>    [junit4]   1>
>    [junit4]   1> TEST: got expected fake exc:a random IOException (_29_1.liv)
>    [junit4]   1> java.io.FileNotFoundException: a random IOException (_29_1.liv)
>    [junit4]   1>        at org.apache.lucene.store.MockDirectoryWrapper.maybeThrowIOExceptionOnOpen(MockDirectoryWrapper.java:458)
>    [junit4]   1>        at org.apache.lucene.store.MockDirectoryWrapper.openInput(MockDirectoryWrapper.java:635)
>    [junit4]   1>        at org.apache.lucene.store.Directory.openChecksumInput(Directory.java:109)
>    [junit4]   1>        at org.apache.lucene.store.MockDirectoryWrapper.openChecksumInput(MockDirectoryWrapper.java:1009)
>    [junit4]   1>        at org.apache.lucene.codecs.simpletext.SimpleTextLiveDocsFormat.readLiveDocs(SimpleTextLiveDocsFormat.java:77)
>    [junit4]   1>        at org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:73)
>    [junit4]   1>        at org.apache.lucene.index.ReadersAndUpdates.getReader(ReadersAndUpdates.java:145)
>    [junit4]   1>        at org.apache.lucene.index.ReadersAndUpdates.getReadOnlyClone(ReadersAndUpdates.java:197)
>    [junit4]   1>        at org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:99)
>    [junit4]   1>        at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:433)
>    [junit4]   1>        at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:86)
>    [junit4]   1>        at org.apache.lucene.index.BaseIndexFileFormatTestCase.testRandomExceptions(BaseIndexFileFormatTestCase.java:457)
>    [junit4]   1>        at org.apache.lucene.index.BaseFieldInfoFormatTestCase.testRandomExceptions(BaseFieldInfoFormatTestCase.java:50)
>    [junit4]   1>        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>    [junit4]   1>        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>    [junit4]   1>        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>    [junit4]   1>        at java.lang.reflect.Method.invoke(Method.java:483)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
>    [junit4]   1>        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
>    [junit4]   1>        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
>    [junit4]   1>        at java.lang.Thread.run(Thread.java:745)
>    [junit4]   1>
>    [junit4]   1> TEST: got expected fake exc:a random IOException (_2o.pst)
>    [junit4]   1> java.io.IOException: a random IOException (_2o.pst)
>    [junit4]   1>        at org.apache.lucene.store.MockDirectoryWrapper.maybeThrowIOExceptionOnOpen(MockDirectoryWrapper.java:456)
>    [junit4]   1>        at org.apache.lucene.store.MockDirectoryWrapper.createOutput(MockDirectoryWrapper.java:548)
>    [junit4]   1>        at org.apache.lucene.store.LockValidatingDirectoryWrapper.createOutput(LockValidatingDirectoryWrapper.java:44)
>    [junit4]   1>        at org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:43)
>    [junit4]   1>        at org.apache.lucene.codecs.simpletext.SimpleTextFieldsWriter.<init>(SimpleTextFieldsWriter.java:52)
>    [junit4]   1>        at org.apache.lucene.codecs.simpletext.SimpleTextPostingsFormat.fieldsConsumer(SimpleTextPostingsFormat.java:45)
>    [junit4]   1>        at org.apache.lucene.index.FreqProxTermsWriter.flush(FreqProxTermsWriter.java:104)
>    [junit4]   1>        at org.apache.lucene.index.DefaultIndexingChain.flush(DefaultIndexingChain.java:112)
>    [junit4]   1>        at org.apache.lucene.index.DocumentsWriterPerThread.flush(DocumentsWriterPerThread.java:423)
>    [junit4]   1>        at org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:510)
>    [junit4]   1>        at org.apache.lucene.index.DocumentsWriter.flushAllThreads(DocumentsWriter.java:622)
>    [junit4]   1>        at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:422)
>    [junit4]   1>        at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:86)
>    [junit4]   1>        at org.apache.lucene.index.BaseIndexFileFormatTestCase.testRandomExceptions(BaseIndexFileFormatTestCase.java:457)
>    [junit4]   1>        at org.apache.lucene.index.BaseFieldInfoFormatTestCase.testRandomExceptions(BaseFieldInfoFormatTestCase.java:50)
>    [junit4]   1>        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>    [junit4]   1>        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>    [junit4]   1>        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>    [junit4]   1>        at java.lang.reflect.Method.invoke(Method.java:483)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
>    [junit4]   1>        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
>    [junit4]   1>        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
>    [junit4]   1>        at java.lang.Thread.run(Thread.java:745)
>    [junit4]   1>
>    [junit4]   1> TEST: got expected fake exc:a random IOException (_2z.pst)
>    [junit4]   1> java.io.IOException: a random IOException (_2z.pst)
>    [junit4]   1>        at org.apache.lucene.store.MockDirectoryWrapper.maybeThrowIOExceptionOnOpen(MockDirectoryWrapper.java:456)
>    [junit4]   1>        at org.apache.lucene.store.MockDirectoryWrapper.openInput(MockDirectoryWrapper.java:635)
>    [junit4]   1>        at org.apache.lucene.codecs.simpletext.SimpleTextFieldsReader.<init>(SimpleTextFieldsReader.java:84)
>    [junit4]   1>        at org.apache.lucene.codecs.simpletext.SimpleTextPostingsFormat.fieldsProducer(SimpleTextPostingsFormat.java:50)
>    [junit4]   1>        at org.apache.lucene.index.SegmentCoreReaders.<init>(SegmentCoreReaders.java:104)
>    [junit4]   1>        at org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:65)
>    [junit4]   1>        at org.apache.lucene.index.ReadersAndUpdates.getReader(ReadersAndUpdates.java:145)
>    [junit4]   1>        at org.apache.lucene.index.ReadersAndUpdates.getReadOnlyClone(ReadersAndUpdates.java:197)
>    [junit4]   1>        at org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:99)
>    [junit4]   1>        at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:433)
>    [junit4]   1>        at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:86)
>    [junit4]   1>        at org.apache.lucene.index.BaseIndexFileFormatTestCase.testRandomExceptions(BaseIndexFileFormatTestCase.java:457)
>    [junit4]   1>        at org.apache.lucene.index.BaseFieldInfoFormatTestCase.testRandomExceptions(BaseFieldInfoFormatTestCase.java:50)
>    [junit4]   1>        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>    [junit4]   1>        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>    [junit4]   1>        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>    [junit4]   1>        at java.lang.reflect.Method.invoke(Method.java:483)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
>    [junit4]   1>        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
>    [junit4]   1>        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
>    [junit4]   1>        at java.lang.Thread.run(Thread.java:745)
>    [junit4]   1>
>    [junit4]   1> TEST: got expected fake exc:a random IOException (_33.fld)
>    [junit4]   1> java.io.IOException: a random IOException (_33.fld)
>    [junit4]   1>        at org.apache.lucene.store.MockDirectoryWrapper.maybeThrowIOExceptionOnOpen(MockDirectoryWrapper.java:456)
>    [junit4]   1>        at org.apache.lucene.store.MockDirectoryWrapper.createOutput(MockDirectoryWrapper.java:548)
>    [junit4]   1>        at org.apache.lucene.store.LockValidatingDirectoryWrapper.createOutput(LockValidatingDirectoryWrapper.java:44)
>    [junit4]   1>        at org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:43)
>    [junit4]   1>        at org.apache.lucene.codecs.simpletext.SimpleTextStoredFieldsWriter.<init>(SimpleTextStoredFieldsWriter.java:69)
>    [junit4]   1>        at org.apache.lucene.codecs.simpletext.SimpleTextStoredFieldsFormat.fieldsWriter(SimpleTextStoredFieldsFormat.java:45)
>    [junit4]   1>        at org.apache.lucene.index.DefaultIndexingChain.initStoredFieldsWriter(DefaultIndexingChain.java:81)
>    [junit4]   1>        at org.apache.lucene.index.DefaultIndexingChain.startStoredFields(DefaultIndexingChain.java:258)
>    [junit4]   1>        at org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:330)
>    [junit4]   1>        at org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:234)
>    [junit4]   1>        at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:457)
>    [junit4]   1>        at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1458)
>    [junit4]   1>        at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1237)
>    [junit4]   1>        at org.apache.lucene.index.BaseIndexFileFormatTestCase.testRandomExceptions(BaseIndexFileFormatTestCase.java:427)
>    [junit4]   1>        at org.apache.lucene.index.BaseFieldInfoFormatTestCase.testRandomExceptions(BaseFieldInfoFormatTestCase.java:50)
>    [junit4]   1>        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>    [junit4]   1>        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>    [junit4]   1>        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>    [junit4]   1>        at java.lang.reflect.Method.invoke(Method.java:483)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
>    [junit4]   1>        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
>    [junit4]   1>        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
>    [junit4]   1>        at java.lang.Thread.run(Thread.java:745)
>    [junit4]   1>
>    [junit4]   1> TEST: got expected fake exc:a random IOException (_3e.pst)
>    [junit4]   1> java.io.IOException: a random IOException (_3e.pst)
>    [junit4]   1>        at org.apache.lucene.store.MockDirectoryWrapper.maybeThrowIOExceptionOnOpen(MockDirectoryWrapper.java:456)
>    [junit4]   1>        at org.apache.lucene.store.MockDirectoryWrapper.createOutput(MockDirectoryWrapper.java:548)
>    [junit4]   1>        at org.apache.lucene.store.LockValidatingDirectoryWrapper.createOutput(LockValidatingDirectoryWrapper.java:44)
>    [junit4]   1>        at org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:43)
>    [junit4]   1>        at org.apache.lucene.codecs.simpletext.SimpleTextFieldsWriter.<init>(SimpleTextFieldsWriter.java:52)
>    [junit4]   1>        at org.apache.lucene.codecs.simpletext.SimpleTextPostingsFormat.fieldsConsumer(SimpleTextPostingsFormat.java:45)
>    [junit4]   1>        at org.apache.lucene.index.FreqProxTermsWriter.flush(FreqProxTermsWriter.java:104)
>    [junit4]   1>        at org.apache.lucene.index.DefaultIndexingChain.flush(DefaultIndexingChain.java:112)
>    [junit4]   1>        at org.apache.lucene.index.DocumentsWriterPerThread.flush(DocumentsWriterPerThread.java:423)
>    [junit4]   1>        at org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:510)
>    [junit4]   1>        at org.apache.lucene.index.DocumentsWriter.flushAllThreads(DocumentsWriter.java:622)
>    [junit4]   1>        at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:422)
>    [junit4]   1>        at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:86)
>    [junit4]   1>        at org.apache
>
> [...truncated too long message...]
>
> LockValidatingDirectoryWrapper.java:44)
>    [junit4]   1>        at org.apache.lucene.codecs.simpletext.SimpleTextSegmentInfoFormat.write(SimpleTextSegmentInfoFormat.java:159)
>    [junit4]   1>        at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4187)
>    [junit4]   1>        at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3639)
>    [junit4]   1>        at org.apache.lucene.index.SerialMergeScheduler.merge(SerialMergeScheduler.java:40)
>    [junit4]   1>        at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:1912)
>    [junit4]   1>        at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:453)
>    [junit4]   1>        at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:86)
>    [junit4]   1>        at org.apache.lucene.index.BaseIndexFileFormatTestCase.testRandomExceptions(BaseIndexFileFormatTestCase.java:457)
>    [junit4]   1>        at org.apache.lucene.index.BaseFieldInfoFormatTestCase.testRandomExceptions(BaseFieldInfoFormatTestCase.java:50)
>    [junit4]   1>        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>    [junit4]   1>        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>    [junit4]   1>        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>    [junit4]   1>        at java.lang.reflect.Method.invoke(Method.java:483)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
>    [junit4]   1>        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
>    [junit4]   1>        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
>    [junit4]   1>        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>    [junit4]   1>        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
>    [junit4]   1>        at java.lang.Thread.run(Thread.java:745)
>    [junit4]   1>
>    [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
>    [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestSimpleTextFieldInfoFormat -Dtests.method=testRandomExceptions -Dtests.seed=A1871D12B75F5C9E -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=sq_AL -Dtests.timezone=CAT -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
>    [junit4] FAILURE 3.05s J1 | TestSimpleTextFieldInfoFormat.testRandomExceptions <<<
>    [junit4]    > Throwable #1: java.lang.AssertionError: hit unexpected FileNotFoundException: file=_3w.si
>    [junit4]    >        at __randomizedtesting.SeedInfo.seed([A1871D12B75F5C9E:C9A873D62951053E]:0)
>    [junit4]    >        at org.apache.lucene.index.IndexFileDeleter.deleteFile(IndexFileDeleter.java:751)
>    [junit4]    >        at org.apache.lucene.index.IndexFileDeleter.deletePendingFiles(IndexFileDeleter.java:528)
>    [junit4]    >        at org.apache.lucene.index.IndexFileDeleter.decRef(IndexFileDeleter.java:636)
>    [junit4]    >        at org.apache.lucene.index.IndexFileDeleter.checkpoint(IndexFileDeleter.java:578)
>    [junit4]    >        at org.apache.lucene.index.IndexWriter.checkpoint(IndexWriter.java:2298)
>    [junit4]    >        at org.apache.lucene.index.IndexWriter.publishFlushedSegment(IndexWriter.java:2356)
>    [junit4]    >        at org.apache.lucene.index.DocumentsWriterFlushQueue$FlushTicket.publishFlushedSegment(DocumentsWriterFlushQueue.java:198)
>    [junit4]    >        at org.apache.lucene.index.DocumentsWriterFlushQueue$FlushTicket.finishFlush(DocumentsWriterFlushQueue.java:213)
>    [junit4]    >        at org.apache.lucene.index.DocumentsWriterFlushQueue$SegmentFlushTicket.publish(DocumentsWriterFlushQueue.java:249)
>    [junit4]    >        at org.apache.lucene.index.DocumentsWriterFlushQueue.innerPurge(DocumentsWriterFlushQueue.java:116)
>    [junit4]    >        at org.apache.lucene.index.DocumentsWriterFlushQueue.forcePurge(DocumentsWriterFlushQueue.java:138)
>    [junit4]    >        at org.apache.lucene.index.DocumentsWriter.flushAllThreads(DocumentsWriter.java:632)
>    [junit4]    >        at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:422)
>    [junit4]    >        at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:86)
>    [junit4]    >        at org.apache.lucene.index.BaseIndexFileFormatTestCase.testRandomExceptions(BaseIndexFileFormatTestCase.java:457)
>    [junit4]    >        at org.apache.lucene.index.BaseFieldInfoFormatTestCase.testRandomExceptions(BaseFieldInfoFormatTestCase.java:50)
>    [junit4]    >        at java.lang.Thread.run(Thread.java:745)
>    [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/build/codecs/test/J1/temp/lucene.codecs.simpletext.TestSimpleTextFieldInfoFormat_A1871D12B75F5C9E-001
>    [junit4]   2> NOTE: test params are: codec=Asserting(Lucene53): {}, docValues:{}, sim=DefaultSimilarity, locale=sq_AL, timezone=CAT
>    [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.8.0_25 (64-bit)/cpus=4,threads=1,free=216720192,total=253231104
>    [junit4]   2> NOTE: All tests run in this JVM: [TestSimpleTextFieldInfoFormat]
>    [junit4] Completed [1/21] on J1 in 5.76s, 11 tests, 1 failure <<< FAILURES!
>
> [...truncated 90 lines...]
> BUILD FAILED
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:533: The following error occurred while executing this line:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:474: The following error occurred while executing this line:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:61: The following error occurred while executing this line:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/extra-targets.xml:39: The following error occurred while executing this line:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/build.xml:469: The following error occurred while executing this line:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/common-build.xml:2156: The following error occurred while executing this line:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/module-build.xml:58: The following error occurred while executing this line:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/common-build.xml:1426: The following error occurred while executing this line:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/common-build.xml:981: There were test failures: 21 suites, 616 tests, 1 failure, 1 ignored (1 assumption)
>
> Total time: 95 minutes 31 seconds
> Build step 'Invoke Ant' marked build as failure
> Archiving artifacts
> No prior successful build to compare, so performing full copy of artifacts
> Recording test results
> Email was triggered for: Failure
> Sending email for trigger: Failure
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


[JENKINS] Lucene-Solr-NightlyTests-trunk - Build # 739 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-trunk/739/

1 tests failed.
REGRESSION:  org.apache.lucene.codecs.simpletext.TestSimpleTextFieldInfoFormat.testRandomExceptions

Error Message:
hit unexpected FileNotFoundException: file=_3w.si

Stack Trace:
java.lang.AssertionError: hit unexpected FileNotFoundException: file=_3w.si
	at __randomizedtesting.SeedInfo.seed([A1871D12B75F5C9E:C9A873D62951053E]:0)
	at org.apache.lucene.index.IndexFileDeleter.deleteFile(IndexFileDeleter.java:751)
	at org.apache.lucene.index.IndexFileDeleter.deletePendingFiles(IndexFileDeleter.java:528)
	at org.apache.lucene.index.IndexFileDeleter.decRef(IndexFileDeleter.java:636)
	at org.apache.lucene.index.IndexFileDeleter.checkpoint(IndexFileDeleter.java:578)
	at org.apache.lucene.index.IndexWriter.checkpoint(IndexWriter.java:2298)
	at org.apache.lucene.index.IndexWriter.publishFlushedSegment(IndexWriter.java:2356)
	at org.apache.lucene.index.DocumentsWriterFlushQueue$FlushTicket.publishFlushedSegment(DocumentsWriterFlushQueue.java:198)
	at org.apache.lucene.index.DocumentsWriterFlushQueue$FlushTicket.finishFlush(DocumentsWriterFlushQueue.java:213)
	at org.apache.lucene.index.DocumentsWriterFlushQueue$SegmentFlushTicket.publish(DocumentsWriterFlushQueue.java:249)
	at org.apache.lucene.index.DocumentsWriterFlushQueue.innerPurge(DocumentsWriterFlushQueue.java:116)
	at org.apache.lucene.index.DocumentsWriterFlushQueue.forcePurge(DocumentsWriterFlushQueue.java:138)
	at org.apache.lucene.index.DocumentsWriter.flushAllThreads(DocumentsWriter.java:632)
	at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:422)
	at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:86)
	at org.apache.lucene.index.BaseIndexFileFormatTestCase.testRandomExceptions(BaseIndexFileFormatTestCase.java:457)
	at org.apache.lucene.index.BaseFieldInfoFormatTestCase.testRandomExceptions(BaseFieldInfoFormatTestCase.java:50)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:483)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 6446 lines...]
   [junit4] Suite: org.apache.lucene.codecs.simpletext.TestSimpleTextFieldInfoFormat
   [junit4]   1> Unexpected exception: dumping fake-exception-log:...
   [junit4]   1> 
   [junit4]   1> TEST: got expected fake exc:a random IOException (_7.si)
   [junit4]   1> java.io.IOException: a random IOException (_7.si)
   [junit4]   1> 	at org.apache.lucene.store.MockDirectoryWrapper.maybeThrowIOExceptionOnOpen(MockDirectoryWrapper.java:456)
   [junit4]   1> 	at org.apache.lucene.store.MockDirectoryWrapper.createOutput(MockDirectoryWrapper.java:548)
   [junit4]   1> 	at org.apache.lucene.store.LockValidatingDirectoryWrapper.createOutput(LockValidatingDirectoryWrapper.java:44)
   [junit4]   1> 	at org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:43)
   [junit4]   1> 	at org.apache.lucene.codecs.simpletext.SimpleTextSegmentInfoFormat.write(SimpleTextSegmentInfoFormat.java:159)
   [junit4]   1> 	at org.apache.lucene.index.DocumentsWriterPerThread.sealFlushedSegment(DocumentsWriterPerThread.java:502)
   [junit4]   1> 	at org.apache.lucene.index.DocumentsWriterPerThread.flush(DocumentsWriterPerThread.java:460)
   [junit4]   1> 	at org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:510)
   [junit4]   1> 	at org.apache.lucene.index.DocumentsWriter.flushAllThreads(DocumentsWriter.java:622)
   [junit4]   1> 	at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:422)
   [junit4]   1> 	at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:86)
   [junit4]   1> 	at org.apache.lucene.index.BaseIndexFileFormatTestCase.testRandomExceptions(BaseIndexFileFormatTestCase.java:457)
   [junit4]   1> 	at org.apache.lucene.index.BaseFieldInfoFormatTestCase.testRandomExceptions(BaseFieldInfoFormatTestCase.java:50)
   [junit4]   1> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   1> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   [junit4]   1> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   1> 	at java.lang.reflect.Method.invoke(Method.java:483)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
   [junit4]   1> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
   [junit4]   1> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   1> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   1> 
   [junit4]   1> TEST: got expected fake exc:a random IOException (_a.pst)
   [junit4]   1> java.io.IOException: a random IOException (_a.pst)
   [junit4]   1> 	at org.apache.lucene.store.MockDirectoryWrapper.maybeThrowIOExceptionOnOpen(MockDirectoryWrapper.java:456)
   [junit4]   1> 	at org.apache.lucene.store.MockDirectoryWrapper.openInput(MockDirectoryWrapper.java:635)
   [junit4]   1> 	at org.apache.lucene.store.FilterDirectory.openInput(FilterDirectory.java:89)
   [junit4]   1> 	at org.apache.lucene.store.FilterDirectory.openInput(FilterDirectory.java:89)
   [junit4]   1> 	at org.apache.lucene.store.FilterDirectory.openInput(FilterDirectory.java:89)
   [junit4]   1> 	at org.apache.lucene.codecs.simpletext.SimpleTextCompoundFormat.write(SimpleTextCompoundFormat.java:179)
   [junit4]   1> 	at org.apache.lucene.index.IndexWriter.createCompoundFile(IndexWriter.java:4652)
   [junit4]   1> 	at org.apache.lucene.index.DocumentsWriterPerThread.sealFlushedSegment(DocumentsWriterPerThread.java:493)
   [junit4]   1> 	at org.apache.lucene.index.DocumentsWriterPerThread.flush(DocumentsWriterPerThread.java:460)
   [junit4]   1> 	at org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:510)
   [junit4]   1> 	at org.apache.lucene.index.DocumentsWriter.flushAllThreads(DocumentsWriter.java:622)
   [junit4]   1> 	at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:422)
   [junit4]   1> 	at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:86)
   [junit4]   1> 	at org.apache.lucene.index.BaseIndexFileFormatTestCase.testRandomExceptions(BaseIndexFileFormatTestCase.java:457)
   [junit4]   1> 	at org.apache.lucene.index.BaseFieldInfoFormatTestCase.testRandomExceptions(BaseFieldInfoFormatTestCase.java:50)
   [junit4]   1> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   1> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   [junit4]   1> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   1> 	at java.lang.reflect.Method.invoke(Method.java:483)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
   [junit4]   1> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
   [junit4]   1> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   1> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   1> 
   [junit4]   1> TEST: got expected fake exc:a random IOException (_e.fld)
   [junit4]   1> java.io.FileNotFoundException: a random IOException (_e.fld)
   [junit4]   1> 	at org.apache.lucene.store.MockDirectoryWrapper.maybeThrowIOExceptionOnOpen(MockDirectoryWrapper.java:458)
   [junit4]   1> 	at org.apache.lucene.store.MockDirectoryWrapper.openInput(MockDirectoryWrapper.java:635)
   [junit4]   1> 	at org.apache.lucene.codecs.simpletext.SimpleTextStoredFieldsReader.<init>(SimpleTextStoredFieldsReader.java:68)
   [junit4]   1> 	at org.apache.lucene.codecs.simpletext.SimpleTextStoredFieldsFormat.fieldsReader(SimpleTextStoredFieldsFormat.java:40)
   [junit4]   1> 	at org.apache.lucene.index.SegmentCoreReaders.<init>(SegmentCoreReaders.java:117)
   [junit4]   1> 	at org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:65)
   [junit4]   1> 	at org.apache.lucene.index.ReadersAndUpdates.getReader(ReadersAndUpdates.java:145)
   [junit4]   1> 	at org.apache.lucene.index.BufferedUpdatesStream$SegmentState.<init>(BufferedUpdatesStream.java:378)
   [junit4]   1> 	at org.apache.lucene.index.BufferedUpdatesStream.openSegmentStates(BufferedUpdatesStream.java:410)
   [junit4]   1> 	at org.apache.lucene.index.BufferedUpdatesStream.applyDeletesAndUpdates(BufferedUpdatesStream.java:255)
   [junit4]   1> 	at org.apache.lucene.index.IndexWriter._mergeInit(IndexWriter.java:3820)
   [junit4]   1> 	at org.apache.lucene.index.IndexWriter.mergeInit(IndexWriter.java:3778)
   [junit4]   1> 	at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3630)
   [junit4]   1> 	at org.apache.lucene.index.SerialMergeScheduler.merge(SerialMergeScheduler.java:40)
   [junit4]   1> 	at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:1912)
   [junit4]   1> 	at org.apache.lucene.index.IndexWriter.doAfterSegmentFlushed(IndexWriter.java:4703)
   [junit4]   1> 	at org.apache.lucene.index.DocumentsWriter$MergePendingEvent.process(DocumentsWriter.java:697)
   [junit4]   1> 	at org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:4729)
   [junit4]   1> 	at org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:4720)
   [junit4]   1> 	at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1459)
   [junit4]   1> 	at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1237)
   [junit4]   1> 	at org.apache.lucene.index.BaseIndexFileFormatTestCase.testRandomExceptions(BaseIndexFileFormatTestCase.java:427)
   [junit4]   1> 	at org.apache.lucene.index.BaseFieldInfoFormatTestCase.testRandomExceptions(BaseFieldInfoFormatTestCase.java:50)
   [junit4]   1> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   1> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   [junit4]   1> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   1> 	at java.lang.reflect.Method.invoke(Method.java:483)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
   [junit4]   1> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
   [junit4]   1> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   1> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   1> 
   [junit4]   1> TEST: got expected fake exc:a random IOException (_n.inf)
   [junit4]   1> java.io.IOException: a random IOException (_n.inf)
   [junit4]   1> 	at org.apache.lucene.store.MockDirectoryWrapper.maybeThrowIOExceptionOnOpen(MockDirectoryWrapper.java:456)
   [junit4]   1> 	at org.apache.lucene.store.MockDirectoryWrapper.openInput(MockDirectoryWrapper.java:635)
   [junit4]   1> 	at org.apache.lucene.store.Directory.openChecksumInput(Directory.java:109)
   [junit4]   1> 	at org.apache.lucene.store.MockDirectoryWrapper.openChecksumInput(MockDirectoryWrapper.java:1009)
   [junit4]   1> 	at org.apache.lucene.codecs.simpletext.SimpleTextFieldInfosFormat.read(SimpleTextFieldInfosFormat.java:71)
   [junit4]   1> 	at org.apache.lucene.index.SegmentCoreReaders.<init>(SegmentCoreReaders.java:99)
   [junit4]   1> 	at org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:65)
   [junit4]   1> 	at org.apache.lucene.index.ReadersAndUpdates.getReader(ReadersAndUpdates.java:145)
   [junit4]   1> 	at org.apache.lucene.index.BufferedUpdatesStream$SegmentState.<init>(BufferedUpdatesStream.java:378)
   [junit4]   1> 	at org.apache.lucene.index.BufferedUpdatesStream.openSegmentStates(BufferedUpdatesStream.java:410)
   [junit4]   1> 	at org.apache.lucene.index.BufferedUpdatesStream.applyDeletesAndUpdates(BufferedUpdatesStream.java:255)
   [junit4]   1> 	at org.apache.lucene.index.IndexWriter._mergeInit(IndexWriter.java:3820)
   [junit4]   1> 	at org.apache.lucene.index.IndexWriter.mergeInit(IndexWriter.java:3778)
   [junit4]   1> 	at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3630)
   [junit4]   1> 	at org.apache.lucene.index.SerialMergeScheduler.merge(SerialMergeScheduler.java:40)
   [junit4]   1> 	at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:1912)
   [junit4]   1> 	at org.apache.lucene.index.IndexWriter.doAfterSegmentFlushed(IndexWriter.java:4703)
   [junit4]   1> 	at org.apache.lucene.index.DocumentsWriter$MergePendingEvent.process(DocumentsWriter.java:697)
   [junit4]   1> 	at org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:4729)
   [junit4]   1> 	at org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:4720)
   [junit4]   1> 	at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1459)
   [junit4]   1> 	at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1237)
   [junit4]   1> 	at org.apache.lucene.index.BaseIndexFileFormatTestCase.testRandomExceptions(BaseIndexFileFormatTestCase.java:427)
   [junit4]   1> 	at org.apache.lucene.index.BaseFieldInfoFormatTestCase.testRandomExceptions(BaseFieldInfoFormatTestCase.java:50)
   [junit4]   1> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   1> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   [junit4]   1> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   1> 	at java.lang.reflect.Method.invoke(Method.java:483)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
   [junit4]   1> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
   [junit4]   1> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   1> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   1> 
   [junit4]   1> TEST: got expected fake exc:a random IOException (_r.fld)
   [junit4]   1> java.io.IOException: a random IOException (_r.fld)
   [junit4]   1> 	at org.apache.lucene.store.MockDirectoryWrapper.maybeThrowIOExceptionOnOpen(MockDirectoryWrapper.java:456)
   [junit4]   1> 	at org.apache.lucene.store.MockDirectoryWrapper.createOutput(MockDirectoryWrapper.java:548)
   [junit4]   1> 	at org.apache.lucene.store.LockValidatingDirectoryWrapper.createOutput(LockValidatingDirectoryWrapper.java:44)
   [junit4]   1> 	at org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:43)
   [junit4]   1> 	at org.apache.lucene.codecs.simpletext.SimpleTextStoredFieldsWriter.<init>(SimpleTextStoredFieldsWriter.java:69)
   [junit4]   1> 	at org.apache.lucene.codecs.simpletext.SimpleTextStoredFieldsFormat.fieldsWriter(SimpleTextStoredFieldsFormat.java:45)
   [junit4]   1> 	at org.apache.lucene.index.DefaultIndexingChain.initStoredFieldsWriter(DefaultIndexingChain.java:81)
   [junit4]   1> 	at org.apache.lucene.index.DefaultIndexingChain.startStoredFields(DefaultIndexingChain.java:258)
   [junit4]   1> 	at org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:330)
   [junit4]   1> 	at org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:234)
   [junit4]   1> 	at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:457)
   [junit4]   1> 	at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1458)
   [junit4]   1> 	at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1237)
   [junit4]   1> 	at org.apache.lucene.index.BaseIndexFileFormatTestCase.testRandomExceptions(BaseIndexFileFormatTestCase.java:427)
   [junit4]   1> 	at org.apache.lucene.index.BaseFieldInfoFormatTestCase.testRandomExceptions(BaseFieldInfoFormatTestCase.java:50)
   [junit4]   1> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   1> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   [junit4]   1> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   1> 	at java.lang.reflect.Method.invoke(Method.java:483)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
   [junit4]   1> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
   [junit4]   1> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   1> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   1> 
   [junit4]   1> TEST: got expected fake exc:a random IOException (_1a_1.liv)
   [junit4]   1> java.nio.file.NoSuchFileException: a random IOException (_1a_1.liv)
   [junit4]   1> 	at org.apache.lucene.store.MockDirectoryWrapper.maybeThrowIOExceptionOnOpen(MockDirectoryWrapper.java:458)
   [junit4]   1> 	at org.apache.lucene.store.MockDirectoryWrapper.createOutput(MockDirectoryWrapper.java:548)
   [junit4]   1> 	at org.apache.lucene.store.LockValidatingDirectoryWrapper.createOutput(LockValidatingDirectoryWrapper.java:44)
   [junit4]   1> 	at org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:43)
   [junit4]   1> 	at org.apache.lucene.codecs.simpletext.SimpleTextLiveDocsFormat.writeLiveDocs(SimpleTextLiveDocsFormat.java:121)
   [junit4]   1> 	at org.apache.lucene.index.DocumentsWriterPerThread.sealFlushedSegment(DocumentsWriterPerThread.java:528)
   [junit4]   1> 	at org.apache.lucene.index.DocumentsWriterPerThread.flush(DocumentsWriterPerThread.java:460)
   [junit4]   1> 	at org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:510)
   [junit4]   1> 	at org.apache.lucene.index.DocumentsWriter.flushAllThreads(DocumentsWriter.java:622)
   [junit4]   1> 	at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:422)
   [junit4]   1> 	at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:86)
   [junit4]   1> 	at org.apache.lucene.index.BaseIndexFileFormatTestCase.testRandomExceptions(BaseIndexFileFormatTestCase.java:457)
   [junit4]   1> 	at org.apache.lucene.index.BaseFieldInfoFormatTestCase.testRandomExceptions(BaseFieldInfoFormatTestCase.java:50)
   [junit4]   1> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   1> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   [junit4]   1> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   1> 	at java.lang.reflect.Method.invoke(Method.java:483)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
   [junit4]   1> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
   [junit4]   1> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   1> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   1> 
   [junit4]   1> TEST: got expected fake exc:a random IOException (_1d.pst)
   [junit4]   1> java.io.FileNotFoundException: a random IOException (_1d.pst)
   [junit4]   1> 	at org.apache.lucene.store.MockDirectoryWrapper.maybeThrowIOExceptionOnOpen(MockDirectoryWrapper.java:458)
   [junit4]   1> 	at org.apache.lucene.store.MockDirectoryWrapper.openInput(MockDirectoryWrapper.java:635)
   [junit4]   1> 	at org.apache.lucene.store.FilterDirectory.openInput(FilterDirectory.java:89)
   [junit4]   1> 	at org.apache.lucene.store.FilterDirectory.openInput(FilterDirectory.java:89)
   [junit4]   1> 	at org.apache.lucene.store.FilterDirectory.openInput(FilterDirectory.java:89)
   [junit4]   1> 	at org.apache.lucene.codecs.simpletext.SimpleTextCompoundFormat.write(SimpleTextCompoundFormat.java:179)
   [junit4]   1> 	at org.apache.lucene.index.IndexWriter.createCompoundFile(IndexWriter.java:4652)
   [junit4]   1> 	at org.apache.lucene.index.DocumentsWriterPerThread.sealFlushedSegment(DocumentsWriterPerThread.java:493)
   [junit4]   1> 	at org.apache.lucene.index.DocumentsWriterPerThread.flush(DocumentsWriterPerThread.java:460)
   [junit4]   1> 	at org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:510)
   [junit4]   1> 	at org.apache.lucene.index.DocumentsWriter.flushAllThreads(DocumentsWriter.java:622)
   [junit4]   1> 	at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:422)
   [junit4]   1> 	at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:86)
   [junit4]   1> 	at org.apache.lucene.index.BaseIndexFileFormatTestCase.testRandomExceptions(BaseIndexFileFormatTestCase.java:457)
   [junit4]   1> 	at org.apache.lucene.index.BaseFieldInfoFormatTestCase.testRandomExceptions(BaseFieldInfoFormatTestCase.java:50)
   [junit4]   1> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   1> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   [junit4]   1> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   1> 	at java.lang.reflect.Method.invoke(Method.java:483)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
   [junit4]   1> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
   [junit4]   1> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   1> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   1> 
   [junit4]   1> TEST: got expected fake exc:a random IOException (_1h.scf)
   [junit4]   1> java.io.IOException: a random IOException (_1h.scf)
   [junit4]   1> 	at org.apache.lucene.store.MockDirectoryWrapper.maybeThrowIOExceptionOnOpen(MockDirectoryWrapper.java:456)
   [junit4]   1> 	at org.apache.lucene.store.MockDirectoryWrapper.openInput(MockDirectoryWrapper.java:635)
   [junit4]   1> 	at org.apache.lucene.codecs.simpletext.SimpleTextCompoundFormat.getCompoundReader(SimpleTextCompoundFormat.java:58)
   [junit4]   1> 	at org.apache.lucene.index.SegmentCoreReaders.<init>(SegmentCoreReaders.java:93)
   [junit4]   1> 	at org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:65)
   [junit4]   1> 	at org.apache.lucene.index.ReadersAndUpdates.getReader(ReadersAndUpdates.java:145)
   [junit4]   1> 	at org.apache.lucene.index.ReadersAndUpdates.getReadOnlyClone(ReadersAndUpdates.java:197)
   [junit4]   1> 	at org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:99)
   [junit4]   1> 	at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:433)
   [junit4]   1> 	at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:86)
   [junit4]   1> 	at org.apache.lucene.index.BaseIndexFileFormatTestCase.testRandomExceptions(BaseIndexFileFormatTestCase.java:457)
   [junit4]   1> 	at org.apache.lucene.index.BaseFieldInfoFormatTestCase.testRandomExceptions(BaseFieldInfoFormatTestCase.java:50)
   [junit4]   1> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   1> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   [junit4]   1> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   1> 	at java.lang.reflect.Method.invoke(Method.java:483)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
   [junit4]   1> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
   [junit4]   1> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   1> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   1> 
   [junit4]   1> TEST: got expected fake exc:a random IOException (_1q.fld)
   [junit4]   1> java.io.IOException: a random IOException (_1q.fld)
   [junit4]   1> 	at org.apache.lucene.store.MockDirectoryWrapper.maybeThrowIOExceptionOnOpen(MockDirectoryWrapper.java:456)
   [junit4]   1> 	at org.apache.lucene.store.MockDirectoryWrapper.openInput(MockDirectoryWrapper.java:635)
   [junit4]   1> 	at org.apache.lucene.store.FilterDirectory.openInput(FilterDirectory.java:89)
   [junit4]   1> 	at org.apache.lucene.store.FilterDirectory.openInput(FilterDirectory.java:89)
   [junit4]   1> 	at org.apache.lucene.store.FilterDirectory.openInput(FilterDirectory.java:89)
   [junit4]   1> 	at org.apache.lucene.codecs.simpletext.SimpleTextCompoundFormat.write(SimpleTextCompoundFormat.java:179)
   [junit4]   1> 	at org.apache.lucene.index.IndexWriter.createCompoundFile(IndexWriter.java:4652)
   [junit4]   1> 	at org.apache.lucene.index.DocumentsWriterPerThread.sealFlushedSegment(DocumentsWriterPerThread.java:493)
   [junit4]   1> 	at org.apache.lucene.index.DocumentsWriterPerThread.flush(DocumentsWriterPerThread.java:460)
   [junit4]   1> 	at org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:510)
   [junit4]   1> 	at org.apache.lucene.index.DocumentsWriter.flushAllThreads(DocumentsWriter.java:622)
   [junit4]   1> 	at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:422)
   [junit4]   1> 	at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:86)
   [junit4]   1> 	at org.apache.lucene.index.BaseIndexFileFormatTestCase.testRandomExceptions(BaseIndexFileFormatTestCase.java:457)
   [junit4]   1> 	at org.apache.lucene.index.BaseFieldInfoFormatTestCase.testRandomExceptions(BaseFieldInfoFormatTestCase.java:50)
   [junit4]   1> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   1> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   [junit4]   1> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   1> 	at java.lang.reflect.Method.invoke(Method.java:483)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
   [junit4]   1> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
   [junit4]   1> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   1> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   1> 
   [junit4]   1> TEST: got expected fake exc:a random IOException (_29.fld)
   [junit4]   1> java.io.IOException: a random IOException (_29.fld)
   [junit4]   1> 	at org.apache.lucene.store.MockDirectoryWrapper.maybeThrowIOExceptionOnOpen(MockDirectoryWrapper.java:456)
   [junit4]   1> 	at org.apache.lucene.store.MockDirectoryWrapper.createOutput(MockDirectoryWrapper.java:548)
   [junit4]   1> 	at org.apache.lucene.store.LockValidatingDirectoryWrapper.createOutput(LockValidatingDirectoryWrapper.java:44)
   [junit4]   1> 	at org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:43)
   [junit4]   1> 	at org.apache.lucene.codecs.simpletext.SimpleTextStoredFieldsWriter.<init>(SimpleTextStoredFieldsWriter.java:69)
   [junit4]   1> 	at org.apache.lucene.codecs.simpletext.SimpleTextStoredFieldsFormat.fieldsWriter(SimpleTextStoredFieldsFormat.java:45)
   [junit4]   1> 	at org.apache.lucene.index.DefaultIndexingChain.initStoredFieldsWriter(DefaultIndexingChain.java:81)
   [junit4]   1> 	at org.apache.lucene.index.DefaultIndexingChain.startStoredFields(DefaultIndexingChain.java:258)
   [junit4]   1> 	at org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:330)
   [junit4]   1> 	at org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:234)
   [junit4]   1> 	at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:457)
   [junit4]   1> 	at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1458)
   [junit4]   1> 	at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1237)
   [junit4]   1> 	at org.apache.lucene.index.BaseIndexFileFormatTestCase.testRandomExceptions(BaseIndexFileFormatTestCase.java:427)
   [junit4]   1> 	at org.apache.lucene.index.BaseFieldInfoFormatTestCase.testRandomExceptions(BaseFieldInfoFormatTestCase.java:50)
   [junit4]   1> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   1> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   [junit4]   1> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   1> 	at java.lang.reflect.Method.invoke(Method.java:483)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
   [junit4]   1> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
   [junit4]   1> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   1> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   1> 
   [junit4]   1> TEST: got expected fake exc:a random IOException (_29_1.liv)
   [junit4]   1> java.io.FileNotFoundException: a random IOException (_29_1.liv)
   [junit4]   1> 	at org.apache.lucene.store.MockDirectoryWrapper.maybeThrowIOExceptionOnOpen(MockDirectoryWrapper.java:458)
   [junit4]   1> 	at org.apache.lucene.store.MockDirectoryWrapper.openInput(MockDirectoryWrapper.java:635)
   [junit4]   1> 	at org.apache.lucene.store.Directory.openChecksumInput(Directory.java:109)
   [junit4]   1> 	at org.apache.lucene.store.MockDirectoryWrapper.openChecksumInput(MockDirectoryWrapper.java:1009)
   [junit4]   1> 	at org.apache.lucene.codecs.simpletext.SimpleTextLiveDocsFormat.readLiveDocs(SimpleTextLiveDocsFormat.java:77)
   [junit4]   1> 	at org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:73)
   [junit4]   1> 	at org.apache.lucene.index.ReadersAndUpdates.getReader(ReadersAndUpdates.java:145)
   [junit4]   1> 	at org.apache.lucene.index.ReadersAndUpdates.getReadOnlyClone(ReadersAndUpdates.java:197)
   [junit4]   1> 	at org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:99)
   [junit4]   1> 	at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:433)
   [junit4]   1> 	at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:86)
   [junit4]   1> 	at org.apache.lucene.index.BaseIndexFileFormatTestCase.testRandomExceptions(BaseIndexFileFormatTestCase.java:457)
   [junit4]   1> 	at org.apache.lucene.index.BaseFieldInfoFormatTestCase.testRandomExceptions(BaseFieldInfoFormatTestCase.java:50)
   [junit4]   1> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   1> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   [junit4]   1> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   1> 	at java.lang.reflect.Method.invoke(Method.java:483)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
   [junit4]   1> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
   [junit4]   1> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   1> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   1> 
   [junit4]   1> TEST: got expected fake exc:a random IOException (_2o.pst)
   [junit4]   1> java.io.IOException: a random IOException (_2o.pst)
   [junit4]   1> 	at org.apache.lucene.store.MockDirectoryWrapper.maybeThrowIOExceptionOnOpen(MockDirectoryWrapper.java:456)
   [junit4]   1> 	at org.apache.lucene.store.MockDirectoryWrapper.createOutput(MockDirectoryWrapper.java:548)
   [junit4]   1> 	at org.apache.lucene.store.LockValidatingDirectoryWrapper.createOutput(LockValidatingDirectoryWrapper.java:44)
   [junit4]   1> 	at org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:43)
   [junit4]   1> 	at org.apache.lucene.codecs.simpletext.SimpleTextFieldsWriter.<init>(SimpleTextFieldsWriter.java:52)
   [junit4]   1> 	at org.apache.lucene.codecs.simpletext.SimpleTextPostingsFormat.fieldsConsumer(SimpleTextPostingsFormat.java:45)
   [junit4]   1> 	at org.apache.lucene.index.FreqProxTermsWriter.flush(FreqProxTermsWriter.java:104)
   [junit4]   1> 	at org.apache.lucene.index.DefaultIndexingChain.flush(DefaultIndexingChain.java:112)
   [junit4]   1> 	at org.apache.lucene.index.DocumentsWriterPerThread.flush(DocumentsWriterPerThread.java:423)
   [junit4]   1> 	at org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:510)
   [junit4]   1> 	at org.apache.lucene.index.DocumentsWriter.flushAllThreads(DocumentsWriter.java:622)
   [junit4]   1> 	at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:422)
   [junit4]   1> 	at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:86)
   [junit4]   1> 	at org.apache.lucene.index.BaseIndexFileFormatTestCase.testRandomExceptions(BaseIndexFileFormatTestCase.java:457)
   [junit4]   1> 	at org.apache.lucene.index.BaseFieldInfoFormatTestCase.testRandomExceptions(BaseFieldInfoFormatTestCase.java:50)
   [junit4]   1> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   1> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   [junit4]   1> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   1> 	at java.lang.reflect.Method.invoke(Method.java:483)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
   [junit4]   1> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
   [junit4]   1> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   1> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   1> 
   [junit4]   1> TEST: got expected fake exc:a random IOException (_2z.pst)
   [junit4]   1> java.io.IOException: a random IOException (_2z.pst)
   [junit4]   1> 	at org.apache.lucene.store.MockDirectoryWrapper.maybeThrowIOExceptionOnOpen(MockDirectoryWrapper.java:456)
   [junit4]   1> 	at org.apache.lucene.store.MockDirectoryWrapper.openInput(MockDirectoryWrapper.java:635)
   [junit4]   1> 	at org.apache.lucene.codecs.simpletext.SimpleTextFieldsReader.<init>(SimpleTextFieldsReader.java:84)
   [junit4]   1> 	at org.apache.lucene.codecs.simpletext.SimpleTextPostingsFormat.fieldsProducer(SimpleTextPostingsFormat.java:50)
   [junit4]   1> 	at org.apache.lucene.index.SegmentCoreReaders.<init>(SegmentCoreReaders.java:104)
   [junit4]   1> 	at org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:65)
   [junit4]   1> 	at org.apache.lucene.index.ReadersAndUpdates.getReader(ReadersAndUpdates.java:145)
   [junit4]   1> 	at org.apache.lucene.index.ReadersAndUpdates.getReadOnlyClone(ReadersAndUpdates.java:197)
   [junit4]   1> 	at org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:99)
   [junit4]   1> 	at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:433)
   [junit4]   1> 	at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:86)
   [junit4]   1> 	at org.apache.lucene.index.BaseIndexFileFormatTestCase.testRandomExceptions(BaseIndexFileFormatTestCase.java:457)
   [junit4]   1> 	at org.apache.lucene.index.BaseFieldInfoFormatTestCase.testRandomExceptions(BaseFieldInfoFormatTestCase.java:50)
   [junit4]   1> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   1> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   [junit4]   1> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   1> 	at java.lang.reflect.Method.invoke(Method.java:483)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
   [junit4]   1> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
   [junit4]   1> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   1> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   1> 
   [junit4]   1> TEST: got expected fake exc:a random IOException (_33.fld)
   [junit4]   1> java.io.IOException: a random IOException (_33.fld)
   [junit4]   1> 	at org.apache.lucene.store.MockDirectoryWrapper.maybeThrowIOExceptionOnOpen(MockDirectoryWrapper.java:456)
   [junit4]   1> 	at org.apache.lucene.store.MockDirectoryWrapper.createOutput(MockDirectoryWrapper.java:548)
   [junit4]   1> 	at org.apache.lucene.store.LockValidatingDirectoryWrapper.createOutput(LockValidatingDirectoryWrapper.java:44)
   [junit4]   1> 	at org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:43)
   [junit4]   1> 	at org.apache.lucene.codecs.simpletext.SimpleTextStoredFieldsWriter.<init>(SimpleTextStoredFieldsWriter.java:69)
   [junit4]   1> 	at org.apache.lucene.codecs.simpletext.SimpleTextStoredFieldsFormat.fieldsWriter(SimpleTextStoredFieldsFormat.java:45)
   [junit4]   1> 	at org.apache.lucene.index.DefaultIndexingChain.initStoredFieldsWriter(DefaultIndexingChain.java:81)
   [junit4]   1> 	at org.apache.lucene.index.DefaultIndexingChain.startStoredFields(DefaultIndexingChain.java:258)
   [junit4]   1> 	at org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:330)
   [junit4]   1> 	at org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:234)
   [junit4]   1> 	at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:457)
   [junit4]   1> 	at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1458)
   [junit4]   1> 	at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1237)
   [junit4]   1> 	at org.apache.lucene.index.BaseIndexFileFormatTestCase.testRandomExceptions(BaseIndexFileFormatTestCase.java:427)
   [junit4]   1> 	at org.apache.lucene.index.BaseFieldInfoFormatTestCase.testRandomExceptions(BaseFieldInfoFormatTestCase.java:50)
   [junit4]   1> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   1> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   [junit4]   1> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   1> 	at java.lang.reflect.Method.invoke(Method.java:483)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
   [junit4]   1> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
   [junit4]   1> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   1> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   1> 
   [junit4]   1> TEST: got expected fake exc:a random IOException (_3e.pst)
   [junit4]   1> java.io.IOException: a random IOException (_3e.pst)
   [junit4]   1> 	at org.apache.lucene.store.MockDirectoryWrapper.maybeThrowIOExceptionOnOpen(MockDirectoryWrapper.java:456)
   [junit4]   1> 	at org.apache.lucene.store.MockDirectoryWrapper.createOutput(MockDirectoryWrapper.java:548)
   [junit4]   1> 	at org.apache.lucene.store.LockValidatingDirectoryWrapper.createOutput(LockValidatingDirectoryWrapper.java:44)
   [junit4]   1> 	at org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:43)
   [junit4]   1> 	at org.apache.lucene.codecs.simpletext.SimpleTextFieldsWriter.<init>(SimpleTextFieldsWriter.java:52)
   [junit4]   1> 	at org.apache.lucene.codecs.simpletext.SimpleTextPostingsFormat.fieldsConsumer(SimpleTextPostingsFormat.java:45)
   [junit4]   1> 	at org.apache.lucene.index.FreqProxTermsWriter.flush(FreqProxTermsWriter.java:104)
   [junit4]   1> 	at org.apache.lucene.index.DefaultIndexingChain.flush(DefaultIndexingChain.java:112)
   [junit4]   1> 	at org.apache.lucene.index.DocumentsWriterPerThread.flush(DocumentsWriterPerThread.java:423)
   [junit4]   1> 	at org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:510)
   [junit4]   1> 	at org.apache.lucene.index.DocumentsWriter.flushAllThreads(DocumentsWriter.java:622)
   [junit4]   1> 	at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:422)
   [junit4]   1> 	at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:86)
   [junit4]   1> 	at org.apache

[...truncated too long message...]

LockValidatingDirectoryWrapper.java:44)
   [junit4]   1> 	at org.apache.lucene.codecs.simpletext.SimpleTextSegmentInfoFormat.write(SimpleTextSegmentInfoFormat.java:159)
   [junit4]   1> 	at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4187)
   [junit4]   1> 	at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3639)
   [junit4]   1> 	at org.apache.lucene.index.SerialMergeScheduler.merge(SerialMergeScheduler.java:40)
   [junit4]   1> 	at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:1912)
   [junit4]   1> 	at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:453)
   [junit4]   1> 	at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:86)
   [junit4]   1> 	at org.apache.lucene.index.BaseIndexFileFormatTestCase.testRandomExceptions(BaseIndexFileFormatTestCase.java:457)
   [junit4]   1> 	at org.apache.lucene.index.BaseFieldInfoFormatTestCase.testRandomExceptions(BaseFieldInfoFormatTestCase.java:50)
   [junit4]   1> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   1> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   [junit4]   1> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   1> 	at java.lang.reflect.Method.invoke(Method.java:483)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
   [junit4]   1> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
   [junit4]   1> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   1> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   1> 
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestSimpleTextFieldInfoFormat -Dtests.method=testRandomExceptions -Dtests.seed=A1871D12B75F5C9E -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=sq_AL -Dtests.timezone=CAT -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
   [junit4] FAILURE 3.05s J1 | TestSimpleTextFieldInfoFormat.testRandomExceptions <<<
   [junit4]    > Throwable #1: java.lang.AssertionError: hit unexpected FileNotFoundException: file=_3w.si
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([A1871D12B75F5C9E:C9A873D62951053E]:0)
   [junit4]    > 	at org.apache.lucene.index.IndexFileDeleter.deleteFile(IndexFileDeleter.java:751)
   [junit4]    > 	at org.apache.lucene.index.IndexFileDeleter.deletePendingFiles(IndexFileDeleter.java:528)
   [junit4]    > 	at org.apache.lucene.index.IndexFileDeleter.decRef(IndexFileDeleter.java:636)
   [junit4]    > 	at org.apache.lucene.index.IndexFileDeleter.checkpoint(IndexFileDeleter.java:578)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.checkpoint(IndexWriter.java:2298)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.publishFlushedSegment(IndexWriter.java:2356)
   [junit4]    > 	at org.apache.lucene.index.DocumentsWriterFlushQueue$FlushTicket.publishFlushedSegment(DocumentsWriterFlushQueue.java:198)
   [junit4]    > 	at org.apache.lucene.index.DocumentsWriterFlushQueue$FlushTicket.finishFlush(DocumentsWriterFlushQueue.java:213)
   [junit4]    > 	at org.apache.lucene.index.DocumentsWriterFlushQueue$SegmentFlushTicket.publish(DocumentsWriterFlushQueue.java:249)
   [junit4]    > 	at org.apache.lucene.index.DocumentsWriterFlushQueue.innerPurge(DocumentsWriterFlushQueue.java:116)
   [junit4]    > 	at org.apache.lucene.index.DocumentsWriterFlushQueue.forcePurge(DocumentsWriterFlushQueue.java:138)
   [junit4]    > 	at org.apache.lucene.index.DocumentsWriter.flushAllThreads(DocumentsWriter.java:632)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:422)
   [junit4]    > 	at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:86)
   [junit4]    > 	at org.apache.lucene.index.BaseIndexFileFormatTestCase.testRandomExceptions(BaseIndexFileFormatTestCase.java:457)
   [junit4]    > 	at org.apache.lucene.index.BaseFieldInfoFormatTestCase.testRandomExceptions(BaseFieldInfoFormatTestCase.java:50)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/build/codecs/test/J1/temp/lucene.codecs.simpletext.TestSimpleTextFieldInfoFormat_A1871D12B75F5C9E-001
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene53): {}, docValues:{}, sim=DefaultSimilarity, locale=sq_AL, timezone=CAT
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.8.0_25 (64-bit)/cpus=4,threads=1,free=216720192,total=253231104
   [junit4]   2> NOTE: All tests run in this JVM: [TestSimpleTextFieldInfoFormat]
   [junit4] Completed [1/21] on J1 in 5.76s, 11 tests, 1 failure <<< FAILURES!

[...truncated 90 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:533: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:474: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:61: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/extra-targets.xml:39: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/build.xml:469: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/common-build.xml:2156: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/module-build.xml:58: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/common-build.xml:1426: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/common-build.xml:981: There were test failures: 21 suites, 616 tests, 1 failure, 1 ignored (1 assumption)

Total time: 95 minutes 31 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
No prior successful build to compare, so performing full copy of artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-NightlyTests-trunk - Build # 738 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-trunk/738/

2 tests failed.
FAILED:  org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=16156, name=collection5, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=16156, name=collection5, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:49602: Could not find collection : awholynewstresscollection_collection5_0
	at __randomizedtesting.SeedInfo.seed([34491A4993AAC335]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:560)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:234)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:226)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:376)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:328)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1086)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:856)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:799)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:894)


REGRESSION:  org.apache.solr.search.function.SortByFunctionTest.test

Error Message:
Exception during query

Stack Trace:
java.lang.RuntimeException: Exception during query
	at __randomizedtesting.SeedInfo.seed([34491A4993AAC335:BC1D25933D56AECD]:0)
	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:763)
	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:730)
	at org.apache.solr.search.function.SortByFunctionTest.test(SortByFunctionTest.java:49)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:483)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: REQUEST FAILED: xpath=//result/doc[1]/int[@name='id'][.='1']
	xml response was: <?xml version="1.0" encoding="UTF-8"?>
<response>
<lst name="responseHeader"><int name="status">0</int><int name="QTime">0</int></lst><result name="response" numFound="4" start="0" maxScore="1.0"><doc><int name="id">3</int><arr name="range_facet_l"><long>3</long></arr><arr name="range_facet_l_dv"><long>3</long></arr><arr name="range_facet_i_dv"><int>3</int></arr><double name="x_td1">3.0</double><double name="y_td1">2.0</double><double name="w_td1">55.0</double><double name="z_td1">5.0</double><arr name="f_t"><str>ipod ipod ipod ipod ipod ipod ipod ipod ipod</str></arr><long name="_version_">1506567504751755264</long><arr name="multiDefault"><str>muLti-Default</str></arr><int name="intDefault">42</int><date name="timestamp">2015-07-13T08:05:35.214Z</date><float name="score">1.0</float></doc><doc><int name="id">4</int><arr name="range_facet_l"><long>4</long></arr><arr name="range_facet_l_dv"><long>4</long></arr><arr name="range_facet_i_dv"><int>4</int></arr><double name="x_td1">4.0</double><double name="y_td1">2.0</double><double name="w_td1">45.0</double><double name="z_td1">5.0</double><arr name="f_t"><str>ipod ipod ipod ipod ipod ipod ipod</str></arr><long name="_version_">1506567504752803840</long><arr name="multiDefault"><str>muLti-Default</str></arr><int name="intDefault">42</int><date name="timestamp">2015-07-13T08:05:35.215Z</date><float name="score">1.0</float></doc><doc><int name="id">1</int><arr name="range_facet_l"><long>1</long></arr><arr name="range_facet_l_dv"><long>1</long></arr><arr name="range_facet_i_dv"><int>1</int></arr><double name="x_td1">0.0</double><double name="y_td1">2.0</double><double name="w_td1">25.0</double><double name="z_td1">5.0</double><arr name="f_t"><str>ipod</str></arr><long name="_version_">1506567504740220928</long><arr name="multiDefault"><str>muLti-Default</str></arr><int name="intDefault">42</int><date name="timestamp">2015-07-13T08:05:35.203Z</date><float name="score">1.0</float></doc><doc><int name="id">2</int><arr name="range_facet_l"><long>2</long></arr><arr name="range_facet_l_dv"><long>2</long></arr><arr name="range_facet_i_dv"><int>2</int></arr><double name="x_td1">2.0</double><double name="y_td1">2.0</double><double name="w_td1">15.0</double><double name="z_td1">5.0</double><arr name="f_t"><str>ipod ipod ipod ipod ipod</str></arr><long name="_version_">1506567504746512384</long><arr name="multiDefault"><str>muLti-Default</str></arr><int name="intDefault">42</int><date name="timestamp">2015-07-13T08:05:35.209Z</date><float name="score">1.0</float></doc></result>
</response>

	request was:q=*:*&fl=*,score&wt=xml
	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:756)
	... 41 more




Build Log:
[...truncated 11025 lines...]
   [junit4] Suite: org.apache.solr.search.function.SortByFunctionTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.search.function.SortByFunctionTest_34491A4993AAC335-001/init-core-data-001
   [junit4]   2> 2251965 INFO  (SUITE-SortByFunctionTest-seed#[34491A4993AAC335]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false)
   [junit4]   2> 2251966 INFO  (SUITE-SortByFunctionTest-seed#[34491A4993AAC335]-worker) [    ] o.a.s.SolrTestCaseJ4 ####initCore
   [junit4]   2> 2251966 INFO  (SUITE-SortByFunctionTest-seed#[34491A4993AAC335]-worker) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/'
   [junit4]   2> 2251967 INFO  (SUITE-SortByFunctionTest-seed#[34491A4993AAC335]-worker) [    ] o.a.s.c.SolrResourceLoader Adding 'file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/lib/.svn/' to classloader
   [junit4]   2> 2251967 INFO  (SUITE-SortByFunctionTest-seed#[34491A4993AAC335]-worker) [    ] o.a.s.c.SolrResourceLoader Adding 'file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/lib/classes/' to classloader
   [junit4]   2> 2251967 INFO  (SUITE-SortByFunctionTest-seed#[34491A4993AAC335]-worker) [    ] o.a.s.c.SolrResourceLoader Adding 'file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/lib/README' to classloader
   [junit4]   2> 2251991 INFO  (SUITE-SortByFunctionTest-seed#[34491A4993AAC335]-worker) [    ] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 2252000 INFO  (SUITE-SortByFunctionTest-seed#[34491A4993AAC335]-worker) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.0.0
   [junit4]   2> 2252021 INFO  (SUITE-SortByFunctionTest-seed#[34491A4993AAC335]-worker) [    ] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 2252021 INFO  (SUITE-SortByFunctionTest-seed#[34491A4993AAC335]-worker) [    ] o.a.s.s.IndexSchema Reading Solr Schema from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/schema.xml
   [junit4]   2> 2252026 INFO  (SUITE-SortByFunctionTest-seed#[34491A4993AAC335]-worker) [    ] o.a.s.s.IndexSchema [null] Schema name=test
   [junit4]   2> 2252121 INFO  (SUITE-SortByFunctionTest-seed#[34491A4993AAC335]-worker) [    ] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 2252126 INFO  (SUITE-SortByFunctionTest-seed#[34491A4993AAC335]-worker) [    ] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 2252127 INFO  (SUITE-SortByFunctionTest-seed#[34491A4993AAC335]-worker) [    ] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 2252131 INFO  (SUITE-SortByFunctionTest-seed#[34491A4993AAC335]-worker) [    ] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 2252132 INFO  (SUITE-SortByFunctionTest-seed#[34491A4993AAC335]-worker) [    ] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 2252134 INFO  (SUITE-SortByFunctionTest-seed#[34491A4993AAC335]-worker) [    ] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 2252134 WARN  (SUITE-SortByFunctionTest-seed#[34491A4993AAC335]-worker) [    ] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 2252134 WARN  (SUITE-SortByFunctionTest-seed#[34491A4993AAC335]-worker) [    ] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 2252134 INFO  (SUITE-SortByFunctionTest-seed#[34491A4993AAC335]-worker) [    ] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 2252135 WARN  (SUITE-SortByFunctionTest-seed#[34491A4993AAC335]-worker) [    ] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 2252135 WARN  (SUITE-SortByFunctionTest-seed#[34491A4993AAC335]-worker) [    ] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 2252135 INFO  (SUITE-SortByFunctionTest-seed#[34491A4993AAC335]-worker) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 2252135 INFO  (SUITE-SortByFunctionTest-seed#[34491A4993AAC335]-worker) [    ] o.a.s.c.SolrResourceLoader using system property solr.solr.home: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr
   [junit4]   2> 2252135 INFO  (SUITE-SortByFunctionTest-seed#[34491A4993AAC335]-worker) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/'
   [junit4]   2> 2252148 INFO  (SUITE-SortByFunctionTest-seed#[34491A4993AAC335]-worker) [    ] o.a.s.c.CoreContainer New CoreContainer 574873837
   [junit4]   2> 2252148 INFO  (SUITE-SortByFunctionTest-seed#[34491A4993AAC335]-worker) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/]
   [junit4]   2> 2252148 INFO  (SUITE-SortByFunctionTest-seed#[34491A4993AAC335]-worker) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/lib
   [junit4]   2> 2252148 WARN  (SUITE-SortByFunctionTest-seed#[34491A4993AAC335]-worker) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/lib).
   [junit4]   2> 2252154 INFO  (SUITE-SortByFunctionTest-seed#[34491A4993AAC335]-worker) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 600000,connTimeout : 60000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 2252156 INFO  (SUITE-SortByFunctionTest-seed#[34491A4993AAC335]-worker) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=30000&connTimeout=30000&retry=true
   [junit4]   2> 2252156 INFO  (SUITE-SortByFunctionTest-seed#[34491A4993AAC335]-worker) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 2252156 INFO  (SUITE-SortByFunctionTest-seed#[34491A4993AAC335]-worker) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 2252156 INFO  (SUITE-SortByFunctionTest-seed#[34491A4993AAC335]-worker) [    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 2252157 INFO  (SUITE-SortByFunctionTest-seed#[34491A4993AAC335]-worker) [    ] o.a.s.c.SolrCore Created CoreDescriptor: {name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, loadOnStartup=true, instanceDir=collection1, collection=collection1, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/, dataDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.search.function.SortByFunctionTest_34491A4993AAC335-001/init-core-data-001, shard=shard1}
   [junit4]   2> 2252158 INFO  (coreLoadExecutor-5515-thread-1) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/'
   [junit4]   2> 2252159 INFO  (coreLoadExecutor-5515-thread-1) [    ] o.a.s.c.SolrResourceLoader Adding 'file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/lib/.svn/' to classloader
   [junit4]   2> 2252159 INFO  (coreLoadExecutor-5515-thread-1) [    ] o.a.s.c.SolrResourceLoader Adding 'file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/lib/classes/' to classloader
   [junit4]   2> 2252159 INFO  (coreLoadExecutor-5515-thread-1) [    ] o.a.s.c.SolrResourceLoader Adding 'file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/lib/README' to classloader
   [junit4]   2> 2252180 INFO  (coreLoadExecutor-5515-thread-1) [    ] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 2252188 INFO  (coreLoadExecutor-5515-thread-1) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.0.0
   [junit4]   2> 2252210 INFO  (coreLoadExecutor-5515-thread-1) [    ] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 2252211 INFO  (coreLoadExecutor-5515-thread-1) [    ] o.a.s.s.IndexSchema Reading Solr Schema from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/schema.xml
   [junit4]   2> 2252217 INFO  (coreLoadExecutor-5515-thread-1) [    ] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 2252325 INFO  (coreLoadExecutor-5515-thread-1) [    ] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 2252331 INFO  (coreLoadExecutor-5515-thread-1) [    ] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 2252334 INFO  (coreLoadExecutor-5515-thread-1) [    ] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 2252338 INFO  (coreLoadExecutor-5515-thread-1) [    ] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 2252340 INFO  (coreLoadExecutor-5515-thread-1) [    ] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 2252341 INFO  (coreLoadExecutor-5515-thread-1) [    ] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 2252341 WARN  (coreLoadExecutor-5515-thread-1) [    ] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 2252342 WARN  (coreLoadExecutor-5515-thread-1) [    ] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 2252342 INFO  (coreLoadExecutor-5515-thread-1) [    ] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 2252342 WARN  (coreLoadExecutor-5515-thread-1) [    ] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 2252342 WARN  (coreLoadExecutor-5515-thread-1) [    ] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 2252342 INFO  (coreLoadExecutor-5515-thread-1) [    ] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/
   [junit4]   2> 2252342 INFO  (coreLoadExecutor-5515-thread-1) [    x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 2252343 INFO  (coreLoadExecutor-5515-thread-1) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/], dataDir=[null]
   [junit4]   2> 2252343 INFO  (coreLoadExecutor-5515-thread-1) [    x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1a219006
   [junit4]   2> 2252343 INFO  (coreLoadExecutor-5515-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.search.function.SortByFunctionTest_34491A4993AAC335-001/init-core-data-001
   [junit4]   2> 2252343 INFO  (coreLoadExecutor-5515-thread-1) [    x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.search.function.SortByFunctionTest_34491A4993AAC335-001/init-core-data-001/index/
   [junit4]   2> 2252343 WARN  (coreLoadExecutor-5515-thread-1) [    x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.search.function.SortByFunctionTest_34491A4993AAC335-001/init-core-data-001/index' doesn't exist. Creating new index...
   [junit4]   2> 2252344 INFO  (coreLoadExecutor-5515-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.search.function.SortByFunctionTest_34491A4993AAC335-001/init-core-data-001/index
   [junit4]   2> 2252344 INFO  (coreLoadExecutor-5515-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=20, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
   [junit4]   2> 2252344 INFO  (coreLoadExecutor-5515-thread-1) [    x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@e64a1ff lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@73d74c20),segFN=segments_1,generation=1}
   [junit4]   2> 2252344 INFO  (coreLoadExecutor-5515-thread-1) [    x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 2252346 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [    x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.search.function.SortByFunctionTest_34491A4993AAC335-001/init-core-data-001/
   [junit4]   2> 2252353 INFO  (coreLoadExecutor-5515-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 2252353 INFO  (coreLoadExecutor-5515-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 2252353 INFO  (coreLoadExecutor-5515-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe-allfields"
   [junit4]   2> 2252353 INFO  (coreLoadExecutor-5515-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
   [junit4]   2> 2252353 INFO  (coreLoadExecutor-5515-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 2252353 INFO  (coreLoadExecutor-5515-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 2252354 INFO  (coreLoadExecutor-5515-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "uniq-fields"
   [junit4]   2> 2252354 INFO  (coreLoadExecutor-5515-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
   [junit4]   2> 2252354 INFO  (coreLoadExecutor-5515-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 2252354 INFO  (coreLoadExecutor-5515-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 2252354 INFO  (coreLoadExecutor-5515-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 2252354 INFO  (coreLoadExecutor-5515-thread-1) [    x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 2252356 INFO  (coreLoadExecutor-5515-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2252357 INFO  (coreLoadExecutor-5515-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2252358 INFO  (coreLoadExecutor-5515-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2252359 INFO  (coreLoadExecutor-5515-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2252362 INFO  (coreLoadExecutor-5515-thread-1) [    x:collection1] o.a.s.c.PluginBag requestHandler : 'lazy' created with startup=lazy 
   [junit4]   2> 2252363 INFO  (coreLoadExecutor-5515-thread-1) [    x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/mbeans,standard,spellCheckCompRH,lazy,/update/json/docs,/admin/luke,spellCheckWithWordbreak_Direct,tvrh,/get,/admin/properties,/update/json,mltrh,/admin/threads,/search-facet-def,/update/csv,/search-facet-invariants,/admin/segments,spellCheckCompRH1,/mlt,/admin/system,/replication,dismax,defaults,/config,/schema,/admin/plugins,/admin/logging,spellCheckWithWordbreak,mock,/admin/ping,/update,/admin/file,/terms,spellCheckCompRH_Direct,/debug/dump
   [junit4]   2> 2252364 WARN  (coreLoadExecutor-5515-thread-1) [    x:collection1] o.a.s.c.SolrCore solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
   [junit4]   2> 2252364 INFO  (coreLoadExecutor-5515-thread-1) [    x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 2252365 INFO  (coreLoadExecutor-5515-thread-1) [    x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 2252365 INFO  (coreLoadExecutor-5515-thread-1) [    x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 2252366 INFO  (coreLoadExecutor-5515-thread-1) [    x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 2252366 INFO  (coreLoadExecutor-5515-thread-1) [    x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 2252366 INFO  (coreLoadExecutor-5515-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=2, maxMergeAtOnceExplicit=5, maxMergedSegmentMB=35.8935546875, floorSegmentMB=1.88671875, forceMergeDeletesPctAllowed=3.3219582670736725, segmentsPerTier=30.0, maxCFSSegmentSizeMB=0.6103515625, noCFSRatio=1.0
   [junit4]   2> 2252367 INFO  (coreLoadExecutor-5515-thread-1) [    x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@e64a1ff lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@73d74c20),segFN=segments_1,generation=1}
   [junit4]   2> 2252367 INFO  (coreLoadExecutor-5515-thread-1) [    x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 2252368 INFO  (coreLoadExecutor-5515-thread-1) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@4fd00e8f[collection1] main
   [junit4]   2> 2252368 INFO  (coreLoadExecutor-5515-thread-1) [    x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {storageIO=org.apache.solr.rest.ManagedResourceStorage$InMemoryStorageIO}
   [junit4]   2> 2252368 INFO  (coreLoadExecutor-5515-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using InMemoryStorage
   [junit4]   2> 2252368 INFO  (coreLoadExecutor-5515-thread-1) [    x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 2252368 INFO  (coreLoadExecutor-5515-thread-1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Initializing spell checkers
   [junit4]   2> 2252378 INFO  (coreLoadExecutor-5515-thread-1) [    x:collection1] o.a.s.s.DirectSolrSpellChecker init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
   [junit4]   2> 2252408 INFO  (coreLoadExecutor-5515-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 2252409 INFO  (searcherExecutor-5516-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: default
   [junit4]   2> 2252410 INFO  (coreLoadExecutor-5515-thread-1) [    x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 2252410 INFO  (coreLoadExecutor-5515-thread-1) [    x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 2252410 INFO  (coreLoadExecutor-5515-thread-1) [    x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 2252410 INFO  (coreLoadExecutor-5515-thread-1) [    x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1506567504726589440
   [junit4]   2> 2252411 INFO  (searcherExecutor-5516-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: direct
   [junit4]   2> 2252411 INFO  (searcherExecutor-5516-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: wordbreak
   [junit4]   2> 2252412 INFO  (searcherExecutor-5516-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: multipleFields
   [junit4]   2> 2252413 INFO  (searcherExecutor-5516-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: jarowinkler
   [junit4]   2> 2252413 INFO  (coreLoadExecutor-5515-thread-1) [    x:collection1] o.a.s.u.UpdateLog Took 3 ms to seed version buckets with highest version 1506567504726589440
   [junit4]   2> 2252413 INFO  (coreLoadExecutor-5515-thread-1) [    x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 2252413 INFO  (searcherExecutor-5516-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: external
   [junit4]   2> 2252415 INFO  (SUITE-SortByFunctionTest-seed#[34491A4993AAC335]-worker) [    ] o.a.s.SolrTestCaseJ4 ####initCore end
   [junit4]   2> 2252421 INFO  (searcherExecutor-5516-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: freq
   [junit4]   2> 2252422 INFO  (searcherExecutor-5516-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: fqcn
   [junit4]   2> 2252423 INFO  (TEST-SortByFunctionTest.test-seed#[34491A4993AAC335]) [    ] o.a.s.SolrTestCaseJ4 ###Starting test
   [junit4]   2> 2252425 INFO  (searcherExecutor-5516-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: perDict
   [junit4]   2> 2252427 INFO  (TEST-SortByFunctionTest.test-seed#[34491A4993AAC335]) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={} {add=[1 (1506567504740220928)]} 0 3
   [junit4]   2> 2252427 INFO  (searcherExecutor-5516-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@4fd00e8f[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2252433 INFO  (TEST-SortByFunctionTest.test-seed#[34491A4993AAC335]) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={} {add=[2 (1506567504746512384)]} 0 4
   [junit4]   2> 2252435 INFO  (TEST-SortByFunctionTest.test-seed#[34491A4993AAC335]) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={} {add=[3 (1506567504751755264)]} 0 0
   [junit4]   2> 2252463 INFO  (TEST-SortByFunctionTest.test-seed#[34491A4993AAC335]) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={} {add=[4 (1506567504752803840)]} 0 28
   [junit4]   2> 2252464 INFO  (TEST-SortByFunctionTest.test-seed#[34491A4993AAC335]) [    ] o.a.s.u.UpdateHandler start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 2252465 INFO  (TEST-SortByFunctionTest.test-seed#[34491A4993AAC335]) [    ] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@e64a1ff lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@73d74c20),segFN=segments_1,generation=1}
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@e64a1ff lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@73d74c20),segFN=segments_2,generation=2}
   [junit4]   2> 2252466 INFO  (TEST-SortByFunctionTest.test-seed#[34491A4993AAC335]) [    ] o.a.s.c.SolrCore newest commit generation = 2
   [junit4]   2> 2252474 INFO  (TEST-SortByFunctionTest.test-seed#[34491A4993AAC335]) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@70ef2384[collection1] main
   [junit4]   2> 2252475 INFO  (searcherExecutor-5516-thread-1) [    ] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@70ef2384[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_2(6.0.0):c4)))}
   [junit4]   2> 2252476 INFO  (TEST-SortByFunctionTest.test-seed#[34491A4993AAC335]) [    ] o.a.s.u.UpdateHandler end_commit_flush
   [junit4]   2> 2252476 INFO  (TEST-SortByFunctionTest.test-seed#[34491A4993AAC335]) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={} {commit=} 0 12
   [junit4]   2> 2252477 INFO  (TEST-SortByFunctionTest.test-seed#[34491A4993AAC335]) [    ] o.a.s.c.S.Request [collection1] webapp=null path=null params={q=*:*&fl=*,score&wt=xml} hits=4 status=0 QTime=0 
   [junit4]   2> 2252478 ERROR (TEST-SortByFunctionTest.test-seed#[34491A4993AAC335]) [    ] o.a.s.SolrTestCaseJ4 REQUEST FAILED: xpath=//result/doc[1]/int[@name='id'][.='1']
   [junit4]   2> 	xml response was: <?xml version="1.0" encoding="UTF-8"?>
   [junit4]   2> <response>
   [junit4]   2> <lst name="responseHeader"><int name="status">0</int><int name="QTime">0</int></lst><result name="response" numFound="4" start="0" maxScore="1.0"><doc><int name="id">3</int><arr name="range_facet_l"><long>3</long></arr><arr name="range_facet_l_dv"><long>3</long></arr><arr name="range_facet_i_dv"><int>3</int></arr><double name="x_td1">3.0</double><double name="y_td1">2.0</double><double name="w_td1">55.0</double><double name="z_td1">5.0</double><arr name="f_t"><str>ipod ipod ipod ipod ipod ipod ipod ipod ipod</str></arr><long name="_version_">1506567504751755264</long><arr name="multiDefault"><str>muLti-Default</str></arr><int name="intDefault">42</int><date name="timestamp">2015-07-13T08:05:35.214Z</date><float name="score">1.0</float></doc><doc><int name="id">4</int><arr name="range_facet_l"><long>4</long></arr><arr name="range_facet_l_dv"><long>4</long></arr><arr name="range_facet_i_dv"><int>4</int></arr><double name="x_td1">4.0</double><double name="y_td1">2.0</double><double name="w_td1">45.0</double><double name="z_td1">5.0</double><arr name="f_t"><str>ipod ipod ipod ipod ipod ipod ipod</str></arr><long name="_version_">1506567504752803840</long><arr name="multiDefault"><str>muLti-Default</str></arr><int name="intDefault">42</int><date name="timestamp">2015-07-13T08:05:35.215Z</date><float name="score">1.0</float></doc><doc><int name="id">1</int><arr name="range_facet_l"><long>1</long></arr><arr name="range_facet_l_dv"><long>1</long></arr><arr name="range_facet_i_dv"><int>1</int></arr><double name="x_td1">0.0</double><double name="y_td1">2.0</double><double name="w_td1">25.0</double><double name="z_td1">5.0</double><arr name="f_t"><str>ipod</str></arr><long name="_version_">1506567504740220928</long><arr name="multiDefault"><str>muLti-Default</str></arr><int name="intDefault">42</int><date name="timestamp">2015-07-13T08:05:35.203Z</date><float name="score">1.0</float></doc><doc><int name="id">2</int><arr name="range_facet_l"><long>2</long></arr><arr name="range_facet_l_dv"><long>2</long></arr><arr name="range_facet_i_dv"><int>2</int></arr><double name="x_td1">2.0</double><double name="y_td1">2.0</double><double name="w_td1">15.0</double><double name="z_td1">5.0</double><arr name="f_t"><str>ipod ipod ipod ipod ipod</str></arr><long name="_version_">1506567504746512384</long><arr name="multiDefault"><str>muLti-Default</str></arr><int name="intDefault">42</int><date name="timestamp">2015-07-13T08:05:35.209Z</date><float name="score">1.0</float></doc></result>
   [junit4]   2> </response>
   [junit4]   2> 
   [junit4]   2> 	request was:q=*:*&fl=*,score&wt=xml
   [junit4]   2> 2252479 ERROR (TEST-SortByFunctionTest.test-seed#[34491A4993AAC335]) [    ] o.a.s.SolrTestCaseJ4 REQUEST FAILED: q=*:*&fl=*,score&wt=xml:java.lang.RuntimeException: REQUEST FAILED: xpath=//result/doc[1]/int[@name='id'][.='1']
   [junit4]   2> 	xml response was: <?xml version="1.0" encoding="UTF-8"?>
   [junit4]   2> <response>
   [junit4]   2> <lst name="responseHeader"><int name="status">0</int><int name="QTime">0</int></lst><result name="response" numFound="4" start="0" maxScore="1.0"><doc><int name="id">3</int><arr name="range_facet_l"><long>3</long></arr><arr name="range_facet_l_dv"><long>3</long></arr><arr name="range_facet_i_dv"><int>3</int></arr><double name="x_td1">3.0</double><double name="y_td1">2.0</double><double name="w_td1">55.0</double><double name="z_td1">5.0</double><arr name="f_t"><str>ipod ipod ipod ipod ipod ipod ipod ipod ipod</str></arr><long name="_version_">1506567504751755264</long><arr name="multiDefault"><str>muLti-Default</str></arr><int name="intDefault">42</int><date name="timestamp">2015-07-13T08:05:35.214Z</date><float name="score">1.0</float></doc><doc><int name="id">4</int><arr name="range_facet_l"><long>4</long></arr><arr name="range_facet_l_dv"><long>4</long></arr><arr name="range_facet_i_dv"><int>4</int></arr><double name="x_td1">4.0</double><double name="y_td1">2.0</double><double name="w_td1">45.0</double><double name="z_td1">5.0</double><arr name="f_t"><str>ipod ipod ipod ipod ipod ipod ipod</str></arr><long name="_version_">1506567504752803840</long><arr name="multiDefault"><str>muLti-Default</str></arr><int name="intDefault">42</int><date name="timestamp">2015-07-13T08:05:35.215Z</date><float name="score">1.0</float></doc><doc><int name="id">1</int><arr name="range_facet_l"><long>1</long></arr><arr name="range_facet_l_dv"><long>1</long></arr><arr name="range_facet_i_dv"><int>1</int></arr><double name="x_td1">0.0</double><double name="y_td1">2.0</double><double name="w_td1">25.0</double><double name="z_td1">5.0</double><arr name="f_t"><str>ipod</str></arr><long name="_version_">1506567504740220928</long><arr name="multiDefault"><str>muLti-Default</str></arr><int name="intDefault">42</int><date name="timestamp">2015-07-13T08:05:35.203Z</date><float name="score">1.0</float></doc><doc><int name="id">2</int><arr name="range_facet_l"><long>2</long></arr><arr name="range_facet_l_dv"><long>2</long></arr><arr name="range_facet_i_dv"><int>2</int></arr><double name="x_td1">2.0</double><double name="y_td1">2.0</double><double name="w_td1">15.0</double><double name="z_td1">5.0</double><arr name="f_t"><str>ipod ipod ipod ipod ipod</str></arr><long name="_version_">1506567504746512384</long><arr name="multiDefault"><str>muLti-Default</str></arr><int name="intDefault">42</int><date name="timestamp">2015-07-13T08:05:35.209Z</date><float name="score">1.0</float></doc></result>
   [junit4]   2> </response>
   [junit4]   2> 
   [junit4]   2> 	request was:q=*:*&fl=*,score&wt=xml
   [junit4]   2> 	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:756)
   [junit4]   2> 	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:730)
   [junit4]   2> 	at org.apache.solr.search.function.SortByFunctionTest.test(SortByFunctionTest.java:49)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   [junit4]   2> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2> 	at java.lang.reflect.Method.invoke(Method.java:483)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
   [junit4]   2> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 
   [junit4]   2> 2252480 INFO  (TEST-SortByFunctionTest.test-seed#[34491A4993AAC335]) [    ] o.a.s.SolrTestCaseJ4 ###Ending test
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=SortByFunctionTest -Dtests.method=test -Dtests.seed=34491A4993AAC335 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=no -Dtests.timezone=America/Cuiaba -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
   [junit4] ERROR   0.07s J1 | SortByFunctionTest.test <<<
   [junit4]    > Throwable #1: java.lang.RuntimeException: Exception during query
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([34491A4993AAC335:BC1D25933D56AECD]:0)
   [junit4]    > 	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:763)
   [junit4]    > 	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:730)
   [junit4]    > 	at org.apache.solr.search.function.SortByFunctionTest.test(SortByFunctionTest.java:49)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]    > Caused by: java.lang.RuntimeException: REQUEST FAILED: xpath=//result/doc[1]/int[@name='id'][.='1']
   [junit4]    > 	xml response was: <?xml version="1.0" encoding="UTF-8"?>
   [junit4]    > <response>
   [junit4]    > <lst name="responseHeader"><int name="status">0</int><int name="QTime">0</int></lst><result name="response" numFound="4" start="0" maxScore="1.0"><doc><int name="id">3</int><arr name="range_facet_l"><long>3</long></arr><arr name="range_facet_l_dv"><long>3</long></arr><arr name="range_facet_i_dv"><int>3</int></arr><double name="x_td1">3.0</double><double name="y_td1">2.0</double><double name="w_td1">55.0</double><double name="z_td1">5.0</double><arr name="f_t"><str>ipod ipod ipod ipod ipod ipod ipod ipod ipod</str></arr><long name="_version_">1506567504751755264</long><arr name="multiDefault"><str>muLti-Default</str></arr><int name="intDefault">42</int><date name="timestamp">2015-07-13T08:05:35.214Z</date><float name="score">1.0</float></doc><doc><int name="id">4</int><arr name="range_facet_l"><long>4</long></arr><arr name="range_facet_l_dv"><long>4</long></arr><arr name="range_facet_i_dv"><int>4</int></arr><double name="x_td1">4.0</double><double name="y_td1">2.0</double><double name="w_td1">45.0</double><double name="z_td1">5.0</double><arr name="f_t"><str>ipod ipod ipod ipod ipod ipod ipod</str></arr><long name="_version_">1506567504752803840</long><arr name="multiDefault"><str>muLti-Default</str></arr><int name="intDefault">42</int><date name="timestamp">2015-07-13T08:05:35.215Z</date><float name="score">1.0</float></doc><doc><int name="id">1</int><arr name="range_facet_l"><long>1</long></arr><arr name="range_facet_l_dv"><long>1</long></arr><arr name="range_facet_i_dv"><int>1</int></arr><double name="x_td1">0.0</double><double name="y_td1">2.0</double><double name="w_td1">25.0</double><double name="z_td1">5.0</double><arr name="f_t"><str>ipod</str></arr><long name="_version_">1506567504740220928</long><arr name="multiDefault"><str>muLti-Default</str></arr><int name="intDefault">42</int><date name="timestamp">2015-07-13T08:05:35.203Z</date><float name="score">1.0</float></doc><doc><int name="id">2</int><arr name="range_facet_l"><long>2</long></arr><arr name="range_facet_l_dv"><long>2</long></arr><arr name="range_facet_i_dv"><int>2</int></arr><double name="x_td1">2.0</double><double name="y_td1">2.0</double><double name="w_td1">15.0</double><double name="z_td1">5.0</double><arr name="f_t"><str>ipod ipod ipod ipod ipod</str></arr><long name="_version_">1506567504746512384</long><arr name="multiDefault"><str>muLti-Default</str></arr><int name="intDefault">42</int><date name="timestamp">2015-07-13T08:05:35.209Z</date><float name="score">1.0</float></doc></result>
   [junit4]    > </response>
   [junit4]    > 	request was:q=*:*&fl=*,score&wt=xml
   [junit4]    > 	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:756)
   [junit4]    > 	... 41 more
   [junit4]   2> 2252492 INFO  (TEST-SortByFunctionTest.testSortJoinDocFreq-seed#[34491A4993AAC335]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testSortJoinDocFreq
   [junit4]   2> 2252494 INFO  (TEST-SortByFunctionTest.testSortJoinDocFreq-seed#[34491A4993AAC335]) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={} {add=[4 (1506567504813621248)]} 0 1
   [junit4]   2> 2252623 INFO  (TEST-SortByFunctionTest.testSortJoinDocFreq-seed#[34491A4993AAC335]) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={} {add=[3 (1506567504817815552)]} 0 128
   [junit4]   2> 2252624 INFO  (TEST-SortByFunctionTest.testSortJoinDocFreq-seed#[34491A4993AAC335]) [    ] o.a.s.u.UpdateHandler start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 2252624 INFO  (TEST-SortByFunctionTest.testSortJoinDocFreq-seed#[34491A4993AAC335]) [    ] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@e64a1ff lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@73d74c20),segFN=segments_2,generation=2}
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@e64a1ff lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@73d74c20),segFN=segments_3,generation=3}
   [junit4]   2> 2252624 INFO  (TEST-SortByFunctionTest.testSortJoinDocFreq-seed#[34491A4993AAC335]) [    ] o.a.s.c.SolrCore newest commit generation = 3
   [junit4]   2> 2252626 INFO  (TEST-SortByFunctionTest.testSortJoinDocFreq-seed#[34491A4993AAC335]) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@383ccf9e[collection1] main
   [junit4]   2> 2252626 INFO  (TEST-SortByFunctionTest.testSortJoinDocFreq-seed#[34491A4993AAC335]) [    ] o.a.s.u.UpdateHandler end_commit_flush
   [junit4]   2> 2252627 INFO  (searcherExecutor-5516-thread-1) [    ] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@383ccf9e[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_4(6.0.0):c4)))}
   [junit4]   2> 2252627 INFO  (TEST-SortByFunctionTest.testSortJoinDocFreq-seed#[34491A4993AAC335]) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={} {commit=} 0 3
   [junit4]   2> 2252629 INFO  (TEST-SortByFunctionTest.testSortJoinDocFreq-seed#[34491A4993AAC335]) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={} {add=[2 (1506567504954130432)]} 0 1
   [junit4]   2> 2252641 INFO  (TEST-SortByFunctionTest.testSortJoinDocFreq-seed#[34491A4993AAC335]) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={} {add=[1 (1506567504956227584)]} 0 11
   [junit4]   2> 2252642 INFO  (TEST-SortByFunctionTest.testSortJoinDocFreq-seed#[34491A4993AAC335]) [    ] o.a.s.u.UpdateHandler start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 2252643 INFO  (TEST-SortByFunctionTest.testSortJoinDocFreq-seed#[34491A4993AAC335]) [    ] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@e64a1ff lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@73d74c20),segFN=segments_3,generation=3}
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@e64a1ff lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@73d74c20),segFN=segments_4,generation=4}
   [junit4]   2> 2252643 INFO  (TEST-SortByFunctionTest.testSortJoinDocFreq-seed#[34491A4993AAC335]) [    ] o.a.s.c.SolrCore newest commit generation = 4
   [junit4]   2> 2252645 INFO  (TEST-SortByFunctionTest.testSortJoinDocFreq-seed#[34491A4993AAC335]) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@5db7631c[collection1] main
   [junit4]   2> 2252645 INFO  (TEST-SortByFunctionTest.testSortJoinDocFreq-seed#[34491A4993AAC335]) [    ] o.a.s.u.UpdateHandler end_commit_flush
   [junit4]   2> 2252646 INFO  (searcherExecutor-5516-thread-1) [    ] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@5db7631c[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_6(6.0.0):c4)))}
   [junit4]   2> 2252646 INFO  (TEST-SortByFunctionTest.testSortJoinDocFreq-seed#[34491A4993AAC335]) [    ] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=null path=null params={} {commit=} 0 4
   [junit4]   2> 2252648 INFO  (TEST-SortByFunctionTest.testSortJoinDocFreq-seed#[34491A4993AAC335]) [    ] o.a.s.c.S.Request [collection1] webapp=null path=null params={q=links_mfacet:B&fl=id&sort=id+asc&wt=xml} hits=2 status=0 QTime=0 
   [junit4]   2> 2252651 INFO  (TEST-SortByFunctionTest.testSortJoinDocFreq-seed#[34491A4993AAC335]) [    ] o.a.s.c.S.Request [collection1] webapp=null path=null params={q=*:*&fl=id&sort=joindf(id_s1,+links_mfacet)+desc&wt=xml} hits=4 status=0 QTime=2 
   [junit4]   2> 2252652 INFO  (TEST-SortByFunctionTest.testSortJoinDocFreq-seed#[34491A4993AAC335]) [    ] o.a.s.c.S.Request [collection1] webapp=null path=null params={q=*:*&fl=id&sort=joindf(id_s1,+links_mfacet)+asc&wt=xml} hits=4 status=0 QTime=0 
   [junit4]   2> 2252653 INFO  (TEST-SortByFunctionTest.testSortJoinDocFreq-seed#[34491A4993AAC335]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testSortJoinDocFreq
   [junit4]   2> 2252653 INFO  (SUITE-SortByFunctionTest-seed#[34491A4993AAC335]-worker) [    ] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> 2252654 INFO  (SUITE-SortByFunctionTest-seed#[34491A4993AAC335]-worker) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=574873837
   [junit4]   2> 2252654 INFO  (SUITE-SortByFunctionTest-seed#[34491A4993AAC335]-worker) [    x:collection1] o.a.s.c.SolrCore [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@1dbb39fb
   [junit4]   2> 2252655 INFO  (SUITE-SortByFunctionTest-seed#[34491A4993AAC335]-worker) [    x:collection1] o.a.s.u.UpdateHandler closing DirectUpdateHandler2{commits=3,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=8,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0,transaction_logs_total_size=726,transaction_logs_total_number=3}
   [junit4]   2> 2252655 INFO  (SUITE-SortByFunctionTest-seed#[34491A4993AAC335]-worker) [    x:collection1] o.a.s.u.SolrCoreState Closing SolrCoreState
   [junit4]   2> 2252655 INFO  (SUITE-SortByFunctionTest-seed#[34491A4993AAC335]-worker) [    x:collection1] o.a.s.u.DefaultSolrCoreState SolrCoreState ref count has reached 0 - closing IndexWriter
   [junit4]   2> 2252655 INFO  (SUITE-SortByFunctionTest-seed#[34491A4993AAC335]-worker) [    x:collection1] o.a.s.u.DefaultSolrCoreState closing IndexWriter with IndexWriterCloser
   [junit4]   2> 2252655 INFO  (SUITE-SortByFunctionTest-seed#[34491A4993AAC335]-worker) [    x:collection1] o.a.s.c.SolrCore [collection1] Closing main searcher on request.
   [junit4]   2> 2252672 INFO  (SUITE-SortByFunctionTest-seed#[34491A4993AAC335]-worker) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing MockDirectoryFactory - 2 directories currently being tracked
   [junit4]   2> 2252672 INFO  (SUITE-SortByFunctionTest-seed#[34491A4993AAC335]-worker) [    x:collection1] o.a.s.c.CachingDirectoryFactory looking to close /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.search.function.SortByFunctionTest_34491A4993AAC335-001/init-core-data-001 [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.search.function.SortByFunctionTest_34491A4993AAC335-001/init-core-data-001;done=false>>]
   [junit4]   2> 2252672 INFO  (SUITE-SortByFunctionTest-seed#[34491A4993AAC335]-worker) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.search.function.SortByFunctionTest_34491A4993AAC335-001/init-core-data-001
   [junit4]   2> 2252672 INFO  (SUITE-SortByFunctionTest-seed#[34491A4993AAC335]-worker) [    x:collection1] o.a.s.c.CachingDirectoryFactory looking to close /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.search.function.SortByFunctionTest_34491A4993AAC335-001/init-core-data-001/index [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.search.function.SortByFunctionTest_34491A4993AAC335-001/init-core-data-001/index;done=false>>]
   [junit4]   2> 2252673 INFO  (SUITE-SortByFunctionTest-seed#[34491A4993AAC335]-worker) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.search.function.SortByFunctionTest_34491A4993AAC335-001/init-core-data-001/index
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.search.function.SortByFunctionTest_34491A4993AAC335-001
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene53): {w_td1=PostingsFormat(name=LuceneVarGapFixedInterval), range_facet_l_dv=BlockTreeOrds(blocksize=128), multiDefault=Lucene50(blocksize=128), intDefault=PostingsFormat(name=Memory doPackFST= true), y_td1=BlockTreeOrds(blocksize=128), range_facet_l=PostingsFormat(name=Memory doPackFST= true), id_s1=PostingsFormat(name=Memory doPackFST= true), _version_=PostingsFormat(name=Memory doPackFST= true), z_td1=Lucene50(blocksize=128), f_t=BlockTreeOrds(blocksize=128), links_mfacet=Lucene50(blocksize=128), id=BlockTreeOrds(blocksize=128), range_facet_i_dv=PostingsFormat(name=Memory doPackFST= true), text=PostingsFormat(name=LuceneVarGapFixedInterval), x_td1=PostingsFormat(name=Memory doPackFST= true), timestamp=PostingsFormat(name=Memory doPackFST= true)}, docValues:{range_facet_l_dv=DocValuesFormat(name=Lucene50), range_facet_i_dv=DocValuesFormat(name=Memory), timestamp=DocValuesFormat(name=Memory)}, sim=DefaultSimilarity, locale=no, timezone=America/Cuiaba
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.8.0_25 (64-bit)/cpus=4,threads=1,free=93223192,total=531628032
   [junit4]   2> NOTE: All tests run in this JVM: [TestFastLRUCache, TestNRTOpen, SpellPossibilityIteratorTest, TestRandomMergePolicy, DocValuesMultiTest, TestSerializedLuceneMatchVersion, SpatialFilterTest, ZkSolrClientTest, TestDefaultSearchFieldResource, CachingDirectoryFactoryTest, TestHashPartitioner, InfoHandlerTest, FieldAnalysisRequestHandlerTest, TestLFUCache, NoCacheHeaderTest, TestJettySolrRunner, TestLeaderElectionZkExpiry, PreAnalyzedFieldTest, TestValueSourceCache, TestCustomDocTransformer, ReturnFieldsTest, TestBulkSchemaConcurrent, CopyFieldTest, TestRTGBase, TestBulkSchemaAPI, TestLMDirichletSimilarityFactory, TestOverriddenPrefixQueryForCustomFieldType, ZkCLITest, SystemInfoHandlerTest, SolrCmdDistributorTest, StatelessScriptUpdateProcessorFactoryTest, TestFoldingMultitermQuery, ConcurrentDeleteAndCreateCollectionTest, TestSQLHandler, TestDistribDocBasedVersion, DistributedExpandComponentTest, ResourceLoaderTest, TestCloudManagedSchema, TestFiltering, HdfsBasicDistributedZkTest, SaslZkACLProviderTest, AddSchemaFieldsUpdateProcessorFactoryTest, TestAuthenticationFramework, TestIndexSearcher, TestCustomSort, JSONWriterTest, TestSolrConfigHandlerCloud, EchoParamsTest, WordBreakSolrSpellCheckerTest, AnalyticsQueryTest, AsyncCallRequestStatusResponseTest, RemoteQueryErrorTest, BlockCacheTest, TestDistributedMissingSort, BasicFunctionalityTest, SuggestComponentTest, TestComplexPhraseQParserPlugin, DistributedQueueTest, LukeRequestHandlerTest, DocumentAnalysisRequestHandlerTest, AnalysisAfterCoreReloadTest, ShowFileRequestHandlerTest, ChaosMonkeySafeLeaderTest, SSLMigrationTest, HdfsRecoverLeaseTest, FileUtilsTest, SpellCheckComponentTest, DistanceFunctionTest, HdfsThreadLeakTest, CoreAdminCreateDiscoverTest, DisMaxRequestHandlerTest, TestConfigSets, CoreAdminRequestStatusTest, BaseCdcrDistributedZkTest, IndexSchemaRuntimeFieldTest, UpdateParamsTest, TestDynamicLoading, TestWriterPerf, SuggesterTSTTest, ShardRoutingCustomTest, TestSolrQueryParserResource, TestExpandComponent, RankQueryTest, BasicDistributedZkTest, OpenCloseCoreStressTest, AutoCommitTest, TestDistributedGrouping, TestFaceting, DistributedSpellCheckComponentTest, TermVectorComponentDistributedTest, ZkControllerTest, TestJoin, TestReload, TestStressVersions, DistributedTermsComponentTest, TestCoreContainer, SimpleFacetsTest, TestSolr4Spatial, SolrCoreTest, PeerSyncTest, ConvertedLegacyTest, TestSort, TestFunctionQuery, DirectUpdateHandlerTest, SoftAutoCommitTest, CurrencyFieldOpenExchangeTest, SolrIndexSplitterTest, SimplePostToolTest, TestCoreDiscovery, TestExtendedDismaxParser, CoreAdminHandlerTest, SuggesterWFSTTest, TestCSVLoader, PolyFieldTest, SchemaVersionSpecificBehaviorTest, SolrCoreCheckLockOnStartupTest, TestUpdate, FieldMutatingUpdateProcessorTest, TestAtomicUpdateErrorCases, DirectUpdateHandlerOptimizeTest, SortByFunctionTest]
   [junit4] Completed [305/515] on J1 in 0.73s, 2 tests, 1 error <<< FAILURES!

[...truncated 670 lines...]
   [junit4] Suite: org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_34491A4993AAC335-001/init-core-data-001
   [junit4]   2> 2838987 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[34491A4993AAC335]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false)
   [junit4]   2> 2838988 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[34491A4993AAC335]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   1> Formatting using clusterid: testClusterID
   [junit4]   2> 2839068 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[34491A4993AAC335]-worker) [    ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
   [junit4]   2> 2839079 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[34491A4993AAC335]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 2839084 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[34491A4993AAC335]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 2839109 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[34491A4993AAC335]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/hdfs to ./temp/Jetty_localhost_40662_hdfs____1xc80/webapp
   [junit4]   2> 2839216 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[34491A4993AAC335]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 2839743 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[34491A4993AAC335]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:40662
   [junit4]   2> 2839823 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[34491A4993AAC335]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 2839824 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[34491A4993AAC335]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 2839838 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[34491A4993AAC335]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_58645_datanode____.b63gbp/webapp
   [junit4]   2> 2839933 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[34491A4993AAC335]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 2840369 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[34491A4993AAC335]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:58645
   [junit4]   2> 2840443 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[34491A4993AAC335]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 2840444 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[34491A4993AAC335]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 2840461 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[34491A4993AAC335]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_40446_datanode____.kpy7a7/webapp
   [junit4]   2> 2840547 INFO  (IPC Server handler 5 on 34693) [    ] BlockStateChange BLOCK* processReport: from storage DS-b6e4da97-7c4f-4360-8904-2434c87a22f6 node DatanodeRegistration(127.0.0.1, datanodeUuid=8ed62380-8e04-4136-89a1-dce532a0bec2, infoPort=58645, ipcPort=49424, storageInfo=lv=-56;cid=testClusterID;nsid=1543738071;c=0), blocks: 0, hasStaleStorages: true, processing time: 0 msecs
   [junit4]   2> 2840548 INFO  (IPC Server handler 5 on 34693) [    ] BlockStateChange BLOCK* processReport: from storage DS-64106c7d-dde5-46e8-ab81-d71cbf6fcf75 node DatanodeRegistration(127.0.0.1, datanodeUuid=8ed62380-8e04-4136-89a1-dce532a0bec2, infoPort=58645, ipcPort=49424, storageInfo=lv=-56;cid=testClusterID;nsid=1543738071;c=0), blocks: 0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 2840603 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[34491A4993AAC335]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 2841204 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[34491A4993AAC335]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:40446
   [junit4]   2> 2841294 INFO  (IPC Server handler 6 on 34693) [    ] BlockStateChange BLOCK* processReport: from storage DS-fe841820-1c17-401f-9875-75d8d462610e node DatanodeRegistration(127.0.0.1, datanodeUuid=593984e5-22c0-40b1-979c-1b0081d0c1d6, infoPort=40446, ipcPort=38240, storageInfo=lv=-56;cid=testClusterID;nsid=1543738071;c=0), blocks: 0, hasStaleStorages: true, processing time: 0 msecs
   [junit4]   2> 2841295 INFO  (IPC Server handler 6 on 34693) [    ] BlockStateChange BLOCK* processReport: from storage DS-f8034be5-fb3a-4125-9c3f-6c7416e512cb node DatanodeRegistration(127.0.0.1, datanodeUuid=593984e5-22c0-40b1-979c-1b0081d0c1d6, infoPort=40446, ipcPort=38240, storageInfo=lv=-56;cid=testClusterID;nsid=1543738071;c=0), blocks: 0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 2841412 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[34491A4993AAC335]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 2841413 INFO  (Thread-6856) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 2841413 INFO  (Thread-6856) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 2841513 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[34491A4993AAC335]) [    ] o.a.s.c.ZkTestServer start zk server on port:42136
   [junit4]   2> 2841513 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[34491A4993AAC335]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2841514 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[34491A4993AAC335]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2841517 INFO  (zkCallback-1470-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1f5523b2 name:ZooKeeperConnection Watcher:127.0.0.1:42136 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2841517 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[34491A4993AAC335]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2841517 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[34491A4993AAC335]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2841518 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[34491A4993AAC335]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 2841523 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[34491A4993AAC335]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2841524 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[34491A4993AAC335]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2841524 INFO  (zkCallback-1471-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4ddf47fa name:ZooKeeperConnection Watcher:127.0.0.1:42136/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2841525 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[34491A4993AAC335]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2841525 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[34491A4993AAC335]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2841525 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[34491A4993AAC335]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 2841526 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[34491A4993AAC335]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 2841528 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[34491A4993AAC335]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 2841529 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[34491A4993AAC335]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 2841530 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[34491A4993AAC335]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 2841530 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[34491A4993AAC335]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 2841532 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[34491A4993AAC335]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 2841532 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[34491A4993AAC335]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 2841534 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[34491A4993AAC335]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 2841534 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[34491A4993AAC335]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 2841536 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[34491A4993AAC335]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 2841536 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[34491A4993AAC335]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 2841537 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[34491A4993AAC335]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 2841538 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[34491A4993AAC335]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 2841539 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[34491A4993AAC335]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 2841540 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[34491A4993AAC335]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 2841541 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[34491A4993AAC335]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 2841541 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[34491A4993AAC335]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 2841543 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[34491A4993AAC335]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 2841543 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[34491A4993AAC335]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 2841545 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[34491A4993AAC335]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 2841545 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[34491A4993AAC335]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 2841546 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[34491A4993AAC335]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 2841547 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[34491A4993AAC335]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 2841548 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[34491A4993AAC335]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 2841548 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[34491A4993AAC335]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 2841831 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[34491A4993AAC335]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_34491A4993AAC335-001/control-001/cores/collection1
   [junit4]   2> 2841833 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[34491A4993AAC335]) [    ] o.e.j.s.Server jetty-9.2.11.v20150529
   [junit4]   2> 2841834 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[34491A4993AAC335]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@3a548cc4{/,null,AVAILABLE}
   [junit4]   2> 2841835 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[34491A4993AAC335]) [    ] o.e.j.s.ServerConnector Started ServerConnector@50b243d7{HTTP/1.1}{127.0.0.1:49602}
   [junit4]   2> 2841835 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[34491A4993AAC335]) [    ] o.e.j.s.Server Started @2845576ms
   [junit4]   2> 2841835 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[34491A4993AAC335]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=hdfs://localhost:34693/hdfs__localhost_34693__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-trunk_solr_build_solr-core_test_J0_temp_solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_34491A4993AAC335-001_tempDir-002_control_data, hostContext=/, hostPort=49602, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_34491A4993AAC335-001/control-001/cores}
   [junit4]   2> 2841835 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[34491A4993AAC335]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@c387f44
   [junit4]   2> 2841835 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[34491A4993AAC335]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_34491A4993AAC335-001/control-001/'
   [junit4]   2> 2841848 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[34491A4993AAC335]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_34491A4993AAC335-001/control-001/solr.xml
   [junit4]   2> 2841853 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[34491A4993AAC335]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_34491A4993AAC335-001/control-001/cores
   [junit4]   2> 2841853 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[34491A4993AAC335]) [    ] o.a.s.c.CoreContainer New CoreContainer 1897585164
   [junit4]   2> 2841853 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[34491A4993AAC335]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_34491A4993AAC335-001/control-001/]
   [junit4]   2> 2841853 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[34491A4993AAC335]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_34491A4993AAC335-001/control-001/lib
   [junit4]   2> 2841854 WARN  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[34491A4993AAC335]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_34491A4993AAC335-001/control-001/lib).
   [junit4]   2> 2841860 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[34491A4993AAC335]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 2841861 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[34491A4993AAC335]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 2841862 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[34491A4993AAC335]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 2841862 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[34491A4993AAC335]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 2841862 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[34491A4993AAC335]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:42136/solr
   [junit4]   2> 2841862 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[34491A4993AAC335]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 2841862 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[34491A4993AAC335]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2841863 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[34491A4993AAC335]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2841864 INFO  (zkCallback-1473-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@138fc8b0 name:ZooKeeperConnection Watcher:127.0.0.1:42136 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2841865 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[34491A4993AAC335]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2841865 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[34491A4993AAC335]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2841867 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[34491A4993AAC335]) [n:127.0.0.1:49602_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2841867 INFO  (zkCallback-1474-thread-1-processing-n:127.0.0.1:49602_) [n:127.0.0.1:49602_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4b37d6b4 name:ZooKeeperConnection Watcher:127.0.0.1:42136/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2841868 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[34491A4993AAC335]) [n:127.0.0.1:49602_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2841869 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[34491A4993AAC335]) [n:127.0.0.1:49602_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 2841870 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[34491A4993AAC335]) [n:127.0.0.1:49602_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 2841872 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[34491A4993AAC335]) [n:127.0.0.1:49602_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 2841874 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[34491A4993AAC335]) [n:127.0.0.1:49602_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 2841876 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[34491A4993AAC335]) [n:127.0.0.1:49602_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 2841878 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[34491A4993AAC335]) [n:127.0.0.1:49602_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 2841879 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[34491A4993AAC335]) [n:127.0.0.1:49602_    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 2841880 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[34491A4993AAC335]) [n:127.0.0.1:49602_    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 2841881 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[34491A4993AAC335]) [n:127.0.0.1:49602_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:49602_
   [junit4]   2> 2841882 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[34491A4993AAC335]) [n:127.0.0.1:49602_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:49602_
   [junit4]   2> 2841884 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[34491A4993AAC335]) [n:127.0.0.1:49602_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 2841885 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[34491A4993AAC335]) [n:127.0.0.1:49602_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 2841887 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[34491A4993AAC335]) [n:127.0.0.1:49602_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 2841887 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[34491A4993AAC335]) [n:127.0.0.1:49602_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94160507627175939-127.0.0.1:49602_-n_0000000000
   [junit4]   2> 2841888 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[34491A4993AAC335]) [n:127.0.0.1:49602_    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:49602_
   [junit4]   2> 2841888 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[34491A4993AAC335]) [n:127.0.0.1:49602_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 2841890 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[34491A4993AAC335]) [n:127.0.0.1:49602_    ] o.a.s.c.Overseer Overseer (id=94160507627175939-127.0.0.1:49602_-n_0000000000) starting
   [junit4]   2> 2841891 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[34491A4993AAC335]) [n:127.0.0.1:49602_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 2841898 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[34491A4993AAC335]) [n:127.0.0.1:49602_    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 2841898 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[34491A4993AAC335]) [n:127.0.0.1:49602_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 2841898 INFO  (OverseerCollectionProcessor-94160507627175939-127.0.0.1:49602_-n_0000000000) [n:127.0.0.1:49602_    ] o.a.s.c.OverseerCollectionProcessor Process current queue of collection creations
   [junit4]   2> 2841898 INFO  (OverseerStateUpdate-94160507627175939-127.0.0.1:49602_-n_0000000000) [n:127.0.0.1:49602_    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 2841902 INFO  (TEST-Hdf

[...truncated too long message...]

.metrics2.util.MBeans.unregister(MBeans.java:81)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.stopMBeans(MetricsSourceAdapter.java:227)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.stop(MetricsSourceAdapter.java:212)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.stopSources(MetricsSystemImpl.java:461)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.stop(MetricsSystemImpl.java:212)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.shutdown(MetricsSystemImpl.java:592)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.shutdownInstance(DefaultMetricsSystem.java:72)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.shutdown(DefaultMetricsSystem.java:68)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.metrics.NameNodeMetrics.shutdown(NameNodeMetrics.java:145)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.NameNode.stop(NameNode.java:822)
   [junit4]   2> 	at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1720)
   [junit4]   2> 	at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1699)
   [junit4]   2> 	at org.apache.solr.cloud.hdfs.HdfsTestUtil.teardownClass(HdfsTestUtil.java:197)
   [junit4]   2> 	at org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.teardownClass(HdfsCollectionsAPIDistributedZkTest.java:50)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   [junit4]   2> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2> 	at java.lang.reflect.Method.invoke(Method.java:483)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:799)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> Caused by: java.lang.NullPointerException
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.blockmanagement.BlocksMap.size(BlocksMap.java:198)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.getTotalBlocks(BlockManager.java:3291)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlocksTotal(FSNamesystem.java:6223)
   [junit4]   2> 	... 53 more
   [junit4]   2> 4168651 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[34491A4993AAC335]-worker) [n:127.0.0.1:57166_ c:addReplicaColl s:shard2 r:core_node6 x:addReplicaColl_shard2_replica3] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> Jul 13, 2015 4:37:31 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 2 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene53): {other_tl1=FSTOrd50, range_facet_l_dv=PostingsFormat(name=LuceneFixedGap), _version_=FSTOrd50, multiDefault=Lucene50(blocksize=128), a_t=FSTOrd50, intDefault=FSTOrd50, a_i1=FSTOrd50, id=PostingsFormat(name=LuceneFixedGap), range_facet_i_dv=FSTOrd50, text=PostingsFormat(name=LuceneVarGapFixedInterval), range_facet_l=FSTOrd50, timestamp=FSTOrd50}, docValues:{range_facet_l_dv=DocValuesFormat(name=Direct), range_facet_i_dv=DocValuesFormat(name=Memory), timestamp=DocValuesFormat(name=Memory)}, sim=DefaultSimilarity, locale=nl_BE, timezone=Africa/Libreville
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.8.0_25 (64-bit)/cpus=4,threads=2,free=176789488,total=531628032
   [junit4]   2> NOTE: All tests run in this JVM: [DebugComponentTest, TestSchemaResource, ChangedSchemaMergeTest, TestDocBasedVersionConstraints, TestSchemaSimilarityResource, BlockDirectoryTest, TestWordDelimiterFilterFactory, TestUniqueKeyFieldResource, DeleteInactiveReplicaTest, SignatureUpdateProcessorFactoryTest, MergeStrategyTest, TestPivotHelperCode, TestRebalanceLeaders, TestNonDefinedSimilarityFactory, TestJsonFacets, TestStressLucene, TestNamedUpdateProcessors, TestFreeTextSuggestions, CSVRequestHandlerTest, ZkStateWriterTest, AssignTest, IgnoreCommitOptimizeUpdateProcessorFactoryTest, BJQParserTest, TestRandomDVFaceting, SharedFSAutoReplicaFailoverTest, TestSchemaManager, TermsComponentTest, ExternalFileFieldSortTest, TestFileDictionaryLookup, TestInfoStreamLogging, TestRTimer, ZkNodePropsTest, ExitableDirectoryReaderTest, DistributedFacetPivotLongTailTest, DistributedFacetPivotWhiteBoxTest, SuggesterFSTTest, HdfsUnloadDistributedZkTest, TestImplicitCoreProperties, SyncSliceTest, TestConfig, DocValuesMissingTest, DateMathParserTest, TestBadConfig, TestSimpleTrackingShardHandler, TestRequestStatusCollectionAPI, TestConfigReload, FastVectorHighlighterTest, DeleteShardTest, TestReloadAndDeleteDocs, CurrencyFieldXmlFileTest, ChaosMonkeyNothingIsSafeTest, DistribDocExpirationUpdateProcessorTest, IndexSchemaTest, TestShardHandlerFactory, DefaultValueUpdateProcessorTest, ConnectionManagerTest, TestTrackingShardHandlerFactory, TestHighlightDedupGrouping, SegmentsInfoRequestHandlerTest, TestIntervalFaceting, HighlighterTest, TestCollectionAPI, RequestHandlersTest, QueryElevationComponentTest, OverseerRolesTest, TestReversedWildcardFilterFactory, TestManagedSchemaFieldTypeResource, TestMergePolicyConfig, TestBlendedInfixSuggestions, TestLazyCores, TestRangeQuery, SolrRequestParserTest, SharedFSAutoReplicaFailoverUtilsTest, TestMacros, TestRealTimeGet, TestCloudInspectUtil, CollectionTooManyReplicasTest, MinimalSchemaTest, TestFieldResource, TestInitQParser, TestBinaryField, TestTrieFacet, CollectionsAPIDistributedZkTest, LeaderElectionIntegrationTest, ShardRoutingTest, BasicZkTest, FullSolrCloudDistribCmdsTest, TestReplicationHandler, SolrTestCaseJ4Test, TestCursorMarkWithoutUniqueKey, TestTolerantSearch, TestEmbeddedSolrServerConstructors, AliasIntegrationTest, CdcrVersionReplicationTest, CleanupOldIndexTest, CustomCollectionTest, LeaderInitiatedRecoveryOnCommitTest, OutOfBoxZkACLAndCredentialsProvidersTest, OverriddenZkACLAndCredentialsProvidersTest, RecoveryAfterSoftCommitTest, ReplicationFactorTest, SimpleCollectionCreateDeleteTest, TestCloudPivotFacet, TestMiniSolrCloudClusterSSL, TestRandomRequestDistribution, TestSolrCloudWithKerberosAlt, TriLevelCompositeIdRoutingTest, HdfsCollectionsAPIDistributedZkTest]
   [junit4] Completed [515/515] on J0 in 1339.51s, 1 test, 1 error <<< FAILURES!

[...truncated 12 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:533: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:474: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:61: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/extra-targets.xml:39: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build.xml:230: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/common-build.xml:513: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/common-build.xml:1426: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/common-build.xml:981: There were test failures: 515 suites, 2025 tests, 2 errors, 20 ignored (10 assumptions)

Total time: 178 minutes 20 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
No prior successful build to compare, so performing full copy of artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-NightlyTests-trunk - Build # 737 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-trunk/737/

1 tests failed.
FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=180040, name=collection5, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=180040, name=collection5, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:45490/le/b: collection already exists: awholynewstresscollection_collection5_3
	at __randomizedtesting.SeedInfo.seed([134041BD5222216C]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:560)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:234)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:226)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:376)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:328)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1086)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:856)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:799)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1572)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1593)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:887)




Build Log:
[...truncated 10650 lines...]
   [junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_134041BD5222216C-001/init-core-data-001
   [junit4]   2> 1470786 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[134041BD5222216C]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false)
   [junit4]   2> 1470786 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[134041BD5222216C]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /le/b
   [junit4]   2> 1470792 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 1470793 INFO  (Thread-176922) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1470793 INFO  (Thread-176922) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 1470893 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.c.ZkTestServer start zk server on port:33148
   [junit4]   2> 1470893 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1470894 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1470899 INFO  (zkCallback-453-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@d856dd5 name:ZooKeeperConnection Watcher:127.0.0.1:33148 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1470899 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1470900 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1470900 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 1470904 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1470906 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1470907 INFO  (zkCallback-454-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1ab50da2 name:ZooKeeperConnection Watcher:127.0.0.1:33148/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1470907 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1470908 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1470908 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 1470910 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 1470912 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 1470913 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 1470915 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 1470915 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 1470918 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 1470918 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 1470920 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1470920 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1470922 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 1470922 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 1470924 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 1470924 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 1470926 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 1470926 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 1470928 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 1470928 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 1470930 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1470931 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1470933 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1470933 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1470935 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 1470935 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 1470936 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 1470937 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 1470939 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1470939 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1470940 INFO  (zkCallback-455-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@76bd2ded name:ZooKeeperConnection Watcher:127.0.0.1:33148 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1470941 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1470941 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1470941 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 1470943 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1470943 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1470944 INFO  (zkCallback-456-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@49ffc0ee name:ZooKeeperConnection Watcher:127.0.0.1:33148/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1470944 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1470945 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1470945 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf2/solrconfig.xml
   [junit4]   2> 1470946 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/solrconfig.xml
   [junit4]   2> 1470948 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf2/schema.xml
   [junit4]   2> 1470948 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/schema.xml
   [junit4]   2> 1470950 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf2/enumsConfig.xml
   [junit4]   2> 1470950 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/enumsConfig.xml
   [junit4]   2> 1470952 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf2/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1470952 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1470954 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf2/stopwords.txt
   [junit4]   2> 1470954 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/stopwords.txt
   [junit4]   2> 1470955 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf2/protwords.txt
   [junit4]   2> 1470956 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/protwords.txt
   [junit4]   2> 1470957 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf2/currency.xml
   [junit4]   2> 1470957 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/currency.xml
   [junit4]   2> 1470959 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf2/open-exchange-rates.json
   [junit4]   2> 1470959 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/open-exchange-rates.json
   [junit4]   2> 1470962 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf2/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1470962 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1470964 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf2/old_synonyms.txt
   [junit4]   2> 1470964 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/old_synonyms.txt
   [junit4]   2> 1470966 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf2/synonyms.txt
   [junit4]   2> 1470966 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/synonyms.txt
   [junit4]   2> 1470968 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/elevate.xml to /configs/conf2/elevate.xml
   [junit4]   2> 1470968 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/elevate.xml
   [junit4]   2> 1471384 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_134041BD5222216C-001/control-001/cores/collection1
   [junit4]   2> 1471392 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.e.j.s.Server jetty-9.2.11.v20150529
   [junit4]   2> 1471399 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@40e6167c{/le/b,null,AVAILABLE}
   [junit4]   2> 1471400 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.e.j.s.ServerConnector Started ServerConnector@55dd74a1{HTTP/1.1}{127.0.0.1:55381}
   [junit4]   2> 1471400 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.e.j.s.Server Started @1475038ms
   [junit4]   2> 1471400 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_134041BD5222216C-001/tempDir-001/control/data, hostContext=/le/b, hostPort=55381, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_134041BD5222216C-001/control-001/cores}
   [junit4]   2> 1471400 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@c387f44
   [junit4]   2> 1471401 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_134041BD5222216C-001/control-001/'
   [junit4]   2> 1471423 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_134041BD5222216C-001/control-001/solr.xml
   [junit4]   2> 1471429 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_134041BD5222216C-001/control-001/cores
   [junit4]   2> 1471429 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.c.CoreContainer New CoreContainer 1973354241
   [junit4]   2> 1471429 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_134041BD5222216C-001/control-001/]
   [junit4]   2> 1471429 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_134041BD5222216C-001/control-001/lib
   [junit4]   2> 1471430 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_134041BD5222216C-001/control-001/lib).
   [junit4]   2> 1471437 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1471439 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1471457 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1471458 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1471458 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:33148/solr
   [junit4]   2> 1471458 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1471458 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1471459 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1471461 INFO  (zkCallback-458-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@18649a2b name:ZooKeeperConnection Watcher:127.0.0.1:33148 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1471466 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1471467 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1471475 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [n:127.0.0.1:55381_le%2Fb    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1471480 INFO  (zkCallback-459-thread-1-processing-n:127.0.0.1:55381_le%2Fb) [n:127.0.0.1:55381_le%2Fb    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@33fa57f8 name:ZooKeeperConnection Watcher:127.0.0.1:33148/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1471480 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [n:127.0.0.1:55381_le%2Fb    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1471482 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [n:127.0.0.1:55381_le%2Fb    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 1471483 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [n:127.0.0.1:55381_le%2Fb    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 1471485 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [n:127.0.0.1:55381_le%2Fb    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 1471487 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [n:127.0.0.1:55381_le%2Fb    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 1471488 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [n:127.0.0.1:55381_le%2Fb    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 1471490 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [n:127.0.0.1:55381_le%2Fb    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 1471491 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [n:127.0.0.1:55381_le%2Fb    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 1471492 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [n:127.0.0.1:55381_le%2Fb    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 1471493 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [n:127.0.0.1:55381_le%2Fb    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:55381_le%2Fb
   [junit4]   2> 1471494 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [n:127.0.0.1:55381_le%2Fb    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:55381_le%2Fb
   [junit4]   2> 1471495 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [n:127.0.0.1:55381_le%2Fb    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 1471496 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [n:127.0.0.1:55381_le%2Fb    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 1471497 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [n:127.0.0.1:55381_le%2Fb    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1471498 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [n:127.0.0.1:55381_le%2Fb    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94154951802355717-127.0.0.1:55381_le%2Fb-n_0000000000
   [junit4]   2> 1471498 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [n:127.0.0.1:55381_le%2Fb    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:55381_le%2Fb
   [junit4]   2> 1471499 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [n:127.0.0.1:55381_le%2Fb    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 1471500 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [n:127.0.0.1:55381_le%2Fb    ] o.a.s.c.Overseer Overseer (id=94154951802355717-127.0.0.1:55381_le%2Fb-n_0000000000) starting
   [junit4]   2> 1471501 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [n:127.0.0.1:55381_le%2Fb    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 1471506 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [n:127.0.0.1:55381_le%2Fb    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 1471509 INFO  (OverseerStateUpdate-94154951802355717-127.0.0.1:55381_le%2Fb-n_0000000000) [n:127.0.0.1:55381_le%2Fb    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 1471519 INFO  (OverseerCollectionProcessor-94154951802355717-127.0.0.1:55381_le%2Fb-n_0000000000) [n:127.0.0.1:55381_le%2Fb    ] o.a.s.c.OverseerCollectionProcessor Process current queue of collection creations
   [junit4]   2> 1471522 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [n:127.0.0.1:55381_le%2Fb    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1471528 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [n:127.0.0.1:55381_le%2Fb    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1471529 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [n:127.0.0.1:55381_le%2Fb    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1471529 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [n:127.0.0.1:55381_le%2Fb    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_134041BD5222216C-001/control-001/cores
   [junit4]   2> 1471531 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [n:127.0.0.1:55381_le%2Fb    ] o.a.s.c.SolrCore Created CoreDescriptor: {name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, loadOnStartup=true, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_134041BD5222216C-001/control-001/cores/collection1, collection=control_collection, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_134041BD5222216C-001/control-001/cores/collection1/, coreNodeName=, dataDir=data/, shard=}
   [junit4]   2> 1471531 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [n:127.0.0.1:55381_le%2Fb    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_134041BD5222216C-001/control-001/cores/collection1/
   [junit4]   2> 1471531 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [n:127.0.0.1:55381_le%2Fb    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 1471547 INFO  (coreLoadExecutor-1686-thread-1-processing-n:127.0.0.1:55381_le%2Fb) [n:127.0.0.1:55381_le%2Fb c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 1471547 INFO  (coreLoadExecutor-1686-thread-1-processing-n:127.0.0.1:55381_le%2Fb) [n:127.0.0.1:55381_le%2Fb c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1471547 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [n:127.0.0.1:55381_le%2Fb    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1
   [junit4]   2> 1471547 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [n:127.0.0.1:55381_le%2Fb    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1471548 INFO  (zkCallback-459-thread-1-processing-n:127.0.0.1:55381_le%2Fb) [n:127.0.0.1:55381_le%2Fb    ] o.a.s.c.DistributedQueue NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 1471548 INFO  (coreLoadExecutor-1686-thread-1-processing-n:127.0.0.1:55381_le%2Fb) [n:127.0.0.1:55381_le%2Fb c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 1471552 INFO  (OverseerStateUpdate-94154951802355717-127.0.0.1:55381_le%2Fb-n_0000000000) [n:127.0.0.1:55381_le%2Fb    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:55381/le/b",
   [junit4]   2>   "node_name":"127.0.0.1:55381_le%2Fb",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 1471553 INFO  (OverseerStateUpdate-94154951802355717-127.0.0.1:55381_le%2Fb-n_0000000000) [n:127.0.0.1:55381_le%2Fb    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:55381/le/b",
   [junit4]   2>   "node_name":"127.0.0.1:55381_le%2Fb",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 1471553 INFO  (OverseerStateUpdate-94154951802355717-127.0.0.1:55381_le%2Fb-n_0000000000) [n:127.0.0.1:55381_le%2Fb    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 1471553 INFO  (OverseerStateUpdate-94154951802355717-127.0.0.1:55381_le%2Fb-n_0000000000) [n:127.0.0.1:55381_le%2Fb    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 1471554 INFO  (zkCallback-459-thread-1-processing-n:127.0.0.1:55381_le%2Fb) [n:127.0.0.1:55381_le%2Fb    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1471556 INFO  (zkCallback-459-thread-1-processing-n:127.0.0.1:55381_le%2Fb) [n:127.0.0.1:55381_le%2Fb    ] o.a.s.c.c.ZkStateReader Updated cluster state version to 1
   [junit4]   2> 1472548 INFO  (coreLoadExecutor-1686-thread-1-processing-n:127.0.0.1:55381_le%2Fb) [n:127.0.0.1:55381_le%2Fb c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1472548 INFO  (coreLoadExecutor-1686-thread-1-processing-n:127.0.0.1:55381_le%2Fb) [n:127.0.0.1:55381_le%2Fb c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 1472549 INFO  (coreLoadExecutor-1686-thread-1-processing-n:127.0.0.1:55381_le%2Fb) [n:127.0.0.1:55381_le%2Fb c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 1472549 INFO  (coreLoadExecutor-1686-thread-1-processing-n:127.0.0.1:55381_le%2Fb) [n:127.0.0.1:55381_le%2Fb c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 1472550 INFO  (coreLoadExecutor-1686-thread-1-processing-n:127.0.0.1:55381_le%2Fb) [n:127.0.0.1:55381_le%2Fb c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1472550 INFO  (coreLoadExecutor-1686-thread-1-processing-n:127.0.0.1:55381_le%2Fb) [n:127.0.0.1:55381_le%2Fb c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_134041BD5222216C-001/control-001/cores/collection1/'
   [junit4]   2> 1472564 INFO  (coreLoadExecutor-1686-thread-1-processing-n:127.0.0.1:55381_le%2Fb) [n:127.0.0.1:55381_le%2Fb c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 1472569 INFO  (coreLoadExecutor-1686-thread-1-processing-n:127.0.0.1:55381_le%2Fb) [n:127.0.0.1:55381_le%2Fb c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 1472572 INFO  (coreLoadExecutor-1686-thread-1-processing-n:127.0.0.1:55381_le%2Fb) [n:127.0.0.1:55381_le%2Fb c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.0.0
   [junit4]   2> 1472580 INFO  (coreLoadExecutor-1686-thread-1-processing-n:127.0.0.1:55381_le%2Fb) [n:127.0.0.1:55381_le%2Fb c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1472582 INFO  (coreLoadExecutor-1686-thread-1-processing-n:127.0.0.1:55381_le%2Fb) [n:127.0.0.1:55381_le%2Fb c:control_collection   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 1472587 INFO  (coreLoadExecutor-1686-thread-1-processing-n:127.0.0.1:55381_le%2Fb) [n:127.0.0.1:55381_le%2Fb c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1472677 INFO  (coreLoadExecutor-1686-thread-1-processing-n:127.0.0.1:55381_le%2Fb) [n:127.0.0.1:55381_le%2Fb c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 1472683 INFO  (coreLoadExecutor-1686-thread-1-processing-n:127.0.0.1:55381_le%2Fb) [n:127.0.0.1:55381_le%2Fb c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 1472684 INFO  (coreLoadExecutor-1686-thread-1-processing-n:127.0.0.1:55381_le%2Fb) [n:127.0.0.1:55381_le%2Fb c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 1472691 INFO  (coreLoadExecutor-1686-thread-1-processing-n:127.0.0.1:55381_le%2Fb) [n:127.0.0.1:55381_le%2Fb c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1472694 INFO  (coreLoadExecutor-1686-thread-1-processing-n:127.0.0.1:55381_le%2Fb) [n:127.0.0.1:55381_le%2Fb c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1472696 INFO  (coreLoadExecutor-1686-thread-1-processing-n:127.0.0.1:55381_le%2Fb) [n:127.0.0.1:55381_le%2Fb c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1472699 WARN  (coreLoadExecutor-1686-thread-1-processing-n:127.0.0.1:55381_le%2Fb) [n:127.0.0.1:55381_le%2Fb c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 1472699 WARN  (coreLoadExecutor-1686-thread-1-processing-n:127.0.0.1:55381_le%2Fb) [n:127.0.0.1:55381_le%2Fb c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 1472699 INFO  (coreLoadExecutor-1686-thread-1-processing-n:127.0.0.1:55381_le%2Fb) [n:127.0.0.1:55381_le%2Fb c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1472700 WARN  (coreLoadExecutor-1686-thread-1-processing-n:127.0.0.1:55381_le%2Fb) [n:127.0.0.1:55381_le%2Fb c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 1472700 WARN  (coreLoadExecutor-1686-thread-1-processing-n:127.0.0.1:55381_le%2Fb) [n:127.0.0.1:55381_le%2Fb c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 1472700 INFO  (coreLoadExecutor-1686-thread-1-processing-n:127.0.0.1:55381_le%2Fb) [n:127.0.0.1:55381_le%2Fb c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 1472700 INFO  (coreLoadExecutor-1686-thread-1-processing-n:127.0.0.1:55381_le%2Fb) [n:127.0.0.1:55381_le%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 1472700 INFO  (coreLoadExecutor-1686-thread-1-processing-n:127.0.0.1:55381_le%2Fb) [n:127.0.0.1:55381_le%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_134041BD5222216C-001/control-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 1472701 INFO  (coreLoadExecutor-1686-thread-1-processing-n:127.0.0.1:55381_le%2Fb) [n:127.0.0.1:55381_le%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@75f05b9e
   [junit4]   2> 1472701 INFO  (coreLoadExecutor-1686-thread-1-processing-n:127.0.0.1:55381_le%2Fb) [n:127.0.0.1:55381_le%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_134041BD5222216C-001/control-001/cores/collection1/data
   [junit4]   2> 1472702 INFO  (coreLoadExecutor-1686-thread-1-processing-n:127.0.0.1:55381_le%2Fb) [n:127.0.0.1:55381_le%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_134041BD5222216C-001/control-001/cores/collection1/data/index/
   [junit4]   2> 1472702 WARN  (coreLoadExecutor-1686-thread-1-processing-n:127.0.0.1:55381_le%2Fb) [n:127.0.0.1:55381_le%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_134041BD5222216C-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 1472702 INFO  (coreLoadExecutor-1686-thread-1-processing-n:127.0.0.1:55381_le%2Fb) [n:127.0.0.1:55381_le%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_134041BD5222216C-001/control-001/cores/collection1/data/index
   [junit4]   2> 1472702 INFO  (coreLoadExecutor-1686-thread-1-processing-n:127.0.0.1:55381_le%2Fb) [n:127.0.0.1:55381_le%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=1559138050, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
   [junit4]   2> 1472703 INFO  (coreLoadExecutor-1686-thread-1-processing-n:127.0.0.1:55381_le%2Fb) [n:127.0.0.1:55381_le%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@494494d5 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@3bd0f27),segFN=segments_1,generation=1}
   [junit4]   2> 1472703 INFO  (coreLoadExecutor-1686-thread-1-processing-n:127.0.0.1:55381_le%2Fb) [n:127.0.0.1:55381_le%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 1472704 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:55381_le%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_134041BD5222216C-001/control-001/cores/collection1/data/
   [junit4]   2> 1472704 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:55381_le%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_134041BD5222216C-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 1472712 INFO  (coreLoadExecutor-1686-thread-1-processing-n:127.0.0.1:55381_le%2Fb) [n:127.0.0.1:55381_le%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 1472713 INFO  (coreLoadExecutor-1686-thread-1-processing-n:127.0.0.1:55381_le%2Fb) [n:127.0.0.1:55381_le%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 1472713 INFO  (coreLoadExecutor-1686-thread-1-processing-n:127.0.0.1:55381_le%2Fb) [n:127.0.0.1:55381_le%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 1472713 INFO  (coreLoadExecutor-1686-thread-1-processing-n:127.0.0.1:55381_le%2Fb) [n:127.0.0.1:55381_le%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1472713 INFO  (coreLoadExecutor-1686-thread-1-processing-n:127.0.0.1:55381_le%2Fb) [n:127.0.0.1:55381_le%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1472713 INFO  (coreLoadExecutor-1686-thread-1-processing-n:127.0.0.1:55381_le%2Fb) [n:127.0.0.1:55381_le%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 1472714 INFO  (coreLoadExecutor-1686-thread-1-processing-n:127.0.0.1:55381_le%2Fb) [n:127.0.0.1:55381_le%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1472714 INFO  (coreLoadExecutor-1686-thread-1-processing-n:127.0.0.1:55381_le%2Fb) [n:127.0.0.1:55381_le%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1472714 INFO  (coreLoadExecutor-1686-thread-1-processing-n:127.0.0.1:55381_le%2Fb) [n:127.0.0.1:55381_le%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 1472716 INFO  (coreLoadExecutor-1686-thread-1-processing-n:127.0.0.1:55381_le%2Fb) [n:127.0.0.1:55381_le%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1472717 INFO  (coreLoadExecutor-1686-thread-1-processing-n:127.0.0.1:55381_le%2Fb) [n:127.0.0.1:55381_le%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1472718 INFO  (coreLoadExecutor-1686-thread-1-processing-n:127.0.0.1:55381_le%2Fb) [n:127.0.0.1:55381_le%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1472719 INFO  (coreLoadExecutor-1686-thread-1-processing-n:127.0.0.1:55381_le%2Fb) [n:127.0.0.1:55381_le%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1472721 INFO  (coreLoadExecutor-1686-thread-1-processing-n:127.0.0.1:55381_le%2Fb) [n:127.0.0.1:55381_le%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/mbeans,standard,/update/csv,/update/json/docs,/admin/luke,/admin/segments,/get,/admin/system,/replication,/admin/properties,/config,/schema,/admin/plugins,/admin/logging,/update/json,/admin/threads,/admin/ping,/update,/admin/file
   [junit4]   2> 1472722 INFO  (coreLoadExecutor-1686-thread-1-processing-n:127.0.0.1:55381_le%2Fb) [n:127.0.0.1:55381_le%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 1472723 INFO  (coreLoadExecutor-1686-thread-1-processing-n:127.0.0.1:55381_le%2Fb) [n:127.0.0.1:55381_le%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1472723 INFO  (coreLoadExecutor-1686-thread-1-processing-n:127.0.0.1:55381_le%2Fb) [n:127.0.0.1:55381_le%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1472724 INFO  (coreLoadExecutor-1686-thread-1-processing-n:127.0.0.1:55381_le%2Fb) [n:127.0.0.1:55381_le%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 1472724 INFO  (coreLoadExecutor-1686-thread-1-processing-n:127.0.0.1:55381_le%2Fb) [n:127.0.0.1:55381_le%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 1472724 INFO  (coreLoadExecutor-1686-thread-1-processing-n:127.0.0.1:55381_le%2Fb) [n:127.0.0.1:55381_le%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=23, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=1.4931640625, noCFSRatio=1.0]
   [junit4]   2> 1472725 INFO  (coreLoadExecutor-1686-thread-1-processing-n:127.0.0.1:55381_le%2Fb) [n:127.0.0.1:55381_le%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@494494d5 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@3bd0f27),segFN=segments_1,generation=1}
   [junit4]   2> 1472725 INFO  (coreLoadExecutor-1686-thread-1-processing-n:127.0.0.1:55381_le%2Fb) [n:127.0.0.1:55381_le%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 1472725 INFO  (coreLoadExecutor-1686-thread-1-processing-n:127.0.0.1:55381_le%2Fb) [n:127.0.0.1:55381_le%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@269723eb[collection1] main
   [junit4]   2> 1472725 INFO  (coreLoadExecutor-1686-thread-1-processing-n:127.0.0.1:55381_le%2Fb) [n:127.0.0.1:55381_le%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 1472726 INFO  (coreLoadExecutor-1686-thread-1-processing-n:127.0.0.1:55381_le%2Fb) [n:127.0.0.1:55381_le%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1472726 INFO  (coreLoadExecutor-1686-thread-1-processing-n:127.0.0.1:55381_le%2Fb) [n:127.0.0.1:55381_le%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 1472727 INFO  (coreLoadExecutor-1686-thread-1-processing-n:127.0.0.1:55381_le%2Fb) [n:127.0.0.1:55381_le%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1472727 INFO  (coreLoadExecutor-1686-thread-1-processing-n:127.0.0.1:55381_le%2Fb) [n:127.0.0.1:55381_le%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 1472727 INFO  (coreLoadExecutor-1686-thread-1-processing-n:127.0.0.1:55381_le%2Fb) [n:127.0.0.1:55381_le%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1472727 INFO  (coreLoadExecutor-1686-thread-1-processing-n:127.0.0.1:55381_le%2Fb) [n:127.0.0.1:55381_le%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 1472728 INFO  (coreLoadExecutor-1686-thread-1-processing-n:127.0.0.1:55381_le%2Fb) [n:127.0.0.1:55381_le%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1472728 INFO  (coreLoadExecutor-1686-thread-1-processing-n:127.0.0.1:55381_le%2Fb) [n:127.0.0.1:55381_le%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 1472728 INFO  (coreLoadExecutor-1686-thread-1-processing-n:127.0.0.1:55381_le%2Fb) [n:127.0.0.1:55381_le%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1472728 INFO  (searcherExecutor-1687-thread-1-processing-n:127.0.0.1:55381_le%2Fb x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:55381_le%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@269723eb[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1472729 INFO  (coreLoadExecutor-1686-thread-1-processing-n:127.0.0.1:55381_le%2Fb) [n:127.0.0.1:55381_le%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 1472729 INFO  (coreLoadExecutor-1686-thread-1-processing-n:127.0.0.1:55381_le%2Fb) [n:127.0.0.1:55381_le%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 1472729 INFO  (coreLoadExecutor-1686-thread-1-processing-n:127.0.0.1:55381_le%2Fb) [n:127.0.0.1:55381_le%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 1472729 INFO  (coreLoadExecutor-1686-thread-1-processing-n:127.0.0.1:55381_le%2Fb) [n:127.0.0.1:55381_le%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1506479230866685952
   [junit4]   2> 1472731 INFO  (coreLoadExecutor-1686-thread-1-processing-n:127.0.0.1:55381_le%2Fb) [n:127.0.0.1:55381_le%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 2 ms to seed version buckets with highest version 1506479230866685952
   [junit4]   2> 1472731 INFO  (coreLoadExecutor-1686-thread-1-processing-n:127.0.0.1:55381_le%2Fb) [n:127.0.0.1:55381_le%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 1472732 INFO  (coreLoadExecutor-1686-thread-1-processing-n:127.0.0.1:55381_le%2Fb) [n:127.0.0.1:55381_le%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 1472732 INFO  (coreZkRegister-1680-thread-1-processing-n:127.0.0.1:55381_le%2Fb x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:55381_le%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:55381/le/b collection:control_collection shard:shard1
   [junit4]   2> 1472733 INFO  (coreZkRegister-1680-thread-1-processing-n:127.0.0.1:55381_le%2Fb x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:55381_le%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 1472736 INFO  (coreZkRegister-1680-thread-1-processing-n:127.0.0.1:55381_le%2Fb x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:55381_le%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94154951802355717-core_node1-n_0000000000
   [junit4]   2> 1472737 INFO  (coreZkRegister-1680-thread-1-processing-n:127.0.0.1:55381_le%2Fb x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:55381_le%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 1472738 INFO  (zkCallback-459-thread-1-processing-n:127.0.0.1:55381_le%2Fb) [n:127.0.0.1:55381_le%2Fb    ] o.a.s.c.DistributedQueue NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 1472739 INFO  (coreZkRegister-1680-thread-1-processing-n:127.0.0.1:55381_le%2Fb x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:55381_le%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1472739 INFO  (coreZkRegister-1680-thread-1-processing-n:127.0.0.1:55381_le%2Fb x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:55381_le%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1472739 INFO  (coreZkRegister-1680-thread-1-processing-n:127.0.0.1:55381_le%2Fb x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:55381_le%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:55381/le/b/collection1/
   [junit4]   2> 1472739 INFO  (OverseerStateUpdate-94154951802355717-127.0.0.1:55381_le%2Fb-n_0000000000) [n:127.0.0.1:55381_le%2Fb    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 1472739 INFO  (coreZkRegister-1680-thread-1-processing-n:127.0.0.1:55381_le%2Fb x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:55381_le%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 1472739 INFO  (coreZkRegister-1680-thread-1-processing-n:127.0.0.1:55381_le%2Fb x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:55381_le%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:55381/le/b/collection1/ has no replicas
   [junit4]   2> 1472740 INFO  (coreZkRegister-1680-thread-1-processing-n:127.0.0.1:55381_le%2Fb x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:55381_le%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:55381/le/b/collection1/ shard1
   [junit4]   2> 1472740 INFO  (coreZkRegister-1680-thread-1-processing-n:127.0.0.1:55381_le%2Fb x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:55381_le%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 1472743 INFO  (zkCallback-459-thread-1-processing-n:127.0.0.1:55381_le%2Fb) [n:127.0.0.1:55381_le%2Fb    ] o.a.s.c.DistributedQueue NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 1472744 INFO  (OverseerStateUpdate-94154951802355717-127.0.0.1:55381_le%2Fb-n_0000000000) [n:127.0.0.1:55381_le%2Fb    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:55381/le/b",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 1
   [junit4]   2> 1472846 INFO  (zkCallback-459-thread-1-processing-n:127.0.0.1:55381_le%2Fb) [n:127.0.0.1:55381_le%2Fb    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1472848 INFO  (zkCallback-459-thread-1-processing-n:127.0.0.1:55381_le%2Fb) [n:127.0.0.1:55381_le%2Fb    ] o.a.s.c.c.ZkStateReader Updated cluster state version to 2
   [junit4]   2> 1472894 INFO  (coreZkRegister-1680-thread-1-processing-n:127.0.0.1:55381_le%2Fb x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:55381_le%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:55381/le/b/collection1/ and leader is http://127.0.0.1:55381/le/b/collection1/
   [junit4]   2> 1472895 INFO  (coreZkRegister-1680-thread-1-processing-n:127.0.0.1:55381_le%2Fb x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:55381_le%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:55381/le/b
   [junit4]   2> 1472895 INFO  (coreZkRegister-1680-thread-1-processing-n:127.0.0.1:55381_le%2Fb x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:55381_le%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1472895 INFO  (coreZkRegister-1680-thread-1-processing-n:127.0.0.1:55381_le%2Fb x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:55381_le%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 1472895 INFO  (coreZkRegister-1680-thread-1-processing-n:127.0.0.1:55381_le%2Fb x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:55381_le%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1472896 INFO  (zkCallback-459-thread-1-processing-n:127.0.0.1:55381_le%2Fb) [n:127.0.0.1:55381_le%2Fb    ] o.a.s.c.DistributedQueue NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 1472897 INFO  (OverseerStateUpdate-94154951802355717-127.0.0.1:55381_le%2Fb-n_0000000000) [n:127.0.0.1:55381_le%2Fb    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:55381/le/b",
   [junit4]   2>   "node_name":"127.0.0.1:55381_le%2Fb",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"} current state version: 2
   [junit4]   2> 1472898 INFO  (OverseerStateUpdate-94154951802355717-127.0.0.1:55381_le%2Fb-n_0000000000) [n:127.0.0.1:55381_le%2Fb    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:55381/le/b",
   [junit4]   2>   "node_name":"127.0.0.1:55381_le%2Fb",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 1473000 INFO  (zkCallback-459-thread-1-processing-n:127.0.0.1:55381_le%2Fb) [n:127.0.0.1:55381_le%2Fb    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1473002 INFO  (zkCallback-459-thread-1-processing-n:127.0.0.1:55381_le%2Fb) [n:127.0.0.1:55381_le%2Fb    ] o.a.s.c.c.ZkStateReader Updated cluster state version to 3
   [junit4]   2> 1473049 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1473050 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1473051 INFO  (zkCallback-461-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@39d5e546 name:ZooKeeperConnection Watcher:127.0.0.1:33148/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1473051 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1473052 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1473052 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1473056 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 1473056 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with stateFormat=2
   [junit4]   2> 1473056 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1473056 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1473057 INFO  (zkCallback-462-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@315397c2 name:ZooKeeperConnection Watcher:127.0.0.1:33148/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1473058 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1473058 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1473059 INFO  (zkCallback-459-thread-1-processing-n:127.0.0.1:55381_le%2Fb) [n:127.0.0.1:55381_le%2Fb    ] o.a.s.c.DistributedQueue NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 1473060 INFO  (OverseerStateUpdate-94154951802355717-127.0.0.1:55381_le%2Fb-n_0000000000) [n:127.0.0.1:55381_le%2Fb    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"create",
   [junit4]   2>   "name":"collection1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "stateFormat":"2"} current state version: 3
   [junit4]   2> 1473060 INFO  (OverseerStateUpdate-94154951802355717-127.0.0.1:55381_le%2Fb-n_0000000000) [n:127.0.0.1:55381_le%2Fb    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 1473163 INFO  (OverseerStateUpdate-94154951802355717-127.0.0.1:55381_le%2Fb-n_0000000000) [n:127.0.0.1:55381_le%2Fb    ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/collection1/state.json
   [junit4]   2> 1473164 INFO  (zkCallback-461-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1473164 INFO  (zkCallback-459-thread-1-processing-n:127.0.0.1:55381_le%2Fb) [n:127.0.0.1:55381_le%2Fb    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1473166 INFO  (zkCallback-461-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated cluster state version to 4
   [junit4]   2> 1473166 INFO  (zkCallback-459-thread-1-processing-n:127.0.0.1:55381_le%2Fb) [n:127.0.0.1:55381_le%2Fb    ] o.a.s.c.c.ZkStateReader Updated cluster state version to 4
   [junit4]   2> 1473361 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_134041BD5222216C-001/shard-1-001/cores/collection1
   [junit4]   2> 1473363 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_134041BD5222216C-001/shard-1-001
   [junit4]   2> 1473363 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.e.j.s.Server jetty-9.2.11.v20150529
   [junit4]   2> 1473365 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@36e4e5f3{/le/b,null,AVAILABLE}
   [junit4]   2> 1473365 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.e.j.s.ServerConnector Started ServerConnector@4e2d30d0{HTTP/1.1}{127.0.0.1:45490}
   [junit4]   2> 1473366 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.e.j.s.Server Started @1477003ms
   [junit4]   2> 1473366 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_134041BD5222216C-001/tempDir-001/jetty1, solrconfig=solrconfig.xml, hostContext=/le/b, hostPort=45490, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_134041BD5222216C-001/shard-1-001/cores}
   [junit4]   2> 1473366 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@c387f44
   [junit4]   2> 1473366 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_134041BD5222216C-001/shard-1-001/'
   [junit4]   2> 1473381 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_134041BD5222216C-001/shard-1-001/solr.xml
   [junit4]   2> 1473387 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_134041BD5222216C-001/shard-1-001/cores
   [junit4]   2> 1473387 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.c.CoreContainer New CoreContainer 1678406124
   [junit4]   2> 1473387 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_134041BD5222216C-001/shard-1-001/]
   [junit4]   2> 1473387 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_134041BD5222216C-001/shard-1-001/lib
   [junit4]   2> 1473387 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_134041BD5222216C-001/shard-1-001/lib).
   [junit4]   2> 1473394 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1473396 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1473396 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1473396 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1473396 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:33148/solr
   [junit4]   2> 1473397 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1473397 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1473398 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1473399 INFO  (zkCallback-464-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@48711f81 name:ZooKeeperConnection Watcher:127.0.0.1:33148 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1473400 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1473400 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1473402 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [n:127.0.0.1:45490_le%2Fb    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1473403 INFO  (zkCallback-465-thread-1-processing-n:127.0.0.1:45490_le%2Fb) [n:127.0.0.1:45490_le%2Fb    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6cbd35c0 name:ZooKeeperConnection Watcher:127.0.0.1:33148/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1473403 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [n:127.0.0.1:45490_le%2Fb    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1473407 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [n:127.0.0.1:45490_le%2Fb    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1474411 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [n:127.0.0.1:45490_le%2Fb    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:45490_le%2Fb
   [junit4]   2> 1474411 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [n:127.0.0.1:45490_le%2Fb    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:45490_le%2Fb
   [junit4]   2> 1474415 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [n:127.0.0.1:45490_le%2Fb    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1474415 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [n:127.0.0.1:45490_le%2Fb    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94154951802355721-127.0.0.1:45490_le%2Fb-n_0000000001
   [junit4]   2> 1474416 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [n:127.0.0.1:45490_le%2Fb    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/94154951802355717-127.0.0.1:55381_le%2Fb-n_0000000000 to know if I could be the leader
   [junit4]   2> 1474416 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [n:127.0.0.1:45490_le%2Fb    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1474417 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [n:127.0.0.1:45490_le%2Fb    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1474417 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [n:127.0.0.1:45490_le%2Fb    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_134041BD5222216C-001/shard-1-001/cores
   [junit4]   2> 1474418 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [n:127.0.0.1:45490_le%2Fb    ] o.a.s.c.SolrCore Created CoreDescriptor: {name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, loadOnStartup=true, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_134041BD5222216C-001/shard-1-001/cores/collection1, collection=collection1, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_134041BD5222216C-001/shard-1-001/cores/collection1/, coreNodeName=, dataDir=data/, shard=}
   [junit4]   2> 1474418 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [n:127.0.0.1:45490_le%2Fb    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_134041BD5222216C-001/shard-1-001/cores/collection1/
   [junit4]   2> 1474418 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [n:127.0.0.1:45490_le%2Fb    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 1474420 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [n:127.0.0.1:45490_le%2Fb    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1
   [junit4]   2> 1474420 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[134041BD5222216C]) [n:127.0.0.1:45490_le%2Fb    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1474421 INFO  (coreLoadExecutor-1697-thread-1-processing-n:127.0.0.1:45490_le%2Fb) [n:127.0.0.1:45490_le%2Fb c:collection1   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 1474422 INFO  (coreLoadExecutor-1697-thread-1-processing-n:127.0.0.1:45490_le%2Fb) [n:127.0.0.1:45490_le%2Fb c:collection1   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1474422 INFO  (zkCallback-459-thread-1-processing-n:127.0.0.1:55381_le%2Fb) [n:127.0.0.1:55381_le%2Fb    ] o.a.s.c.DistributedQueue NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 1474423 INFO  (coreLoadExecutor-1697-thread-1-processing-n:127.0.0.1:45490_le%2Fb) [n:127.0.0.1:45490_le%2Fb c:collection1   x:collection1] o.a.s.c.ZkController Registering watch for external collection collection1
   [junit4]   2> 1474423 INFO  (coreLoadExecutor-1697-thread-1-processing-n:127.0.0.1:45490_le%2Fb) [n:127.0.0.1:45490_le%2Fb c:collection1   x:collection1] o.a.s.c.c.ZkStateReader addZkWatch collection1
   [junit4]   2> 1474423 INFO  (coreLoadExecutor-1697-thread-1-processing-n:127.0.0.1:45490_le%2Fb) [n:127.0.0.1:45490_le%2Fb c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Updating collection state at /collections/collection1/state.json from ZooKeeper... 
   [junit4]   2> 1474423 INFO  (OverseerStateUpdate-94154951802355717-127.0.0.1:55381_le%2Fb-n_0000000000) [n:127.0.0.1:55381_le%2Fb    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:45490/le/b",
   [junit4]   2>   "node_name":"127.0.0.1:45490_le%2Fb",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "operation":"state"} current state version: 4
   [junit4]   2> 1474424 INFO  (OverseerStateUpdate-94154951802355717-127.0.0.1:55381_le%2Fb-n_0000000000) [n:127.0.0.1:55381_le%2Fb    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:45490/le/b",
   [junit4]   2>   "node_name":"127.0.0.1:45490_le%2Fb",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 1474424 INFO  (OverseerStateUpdate-94154951802355717-127.0.0.1:55381_le%2Fb-n_0000000000) [n:127.0.0.1:55381_le%2Fb    ] o.a.s.c.o.ReplicaMutator Collection already exists with numShards=2
   [junit4]   2> 1474424 INFO  (OverseerStateUpdate-94154951802355717-127.0.0.1:55381_le%2Fb-n_0000000000) [n:127.0.0.1:55381_le%2Fb    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2
   [junit4]   2> 1474424 INFO  (coreLoadExecutor-1697-thread-1-processing-n:127.0.0.1:45490_le%2Fb) [n:127.0.0.1:45490_le%2Fb c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Updating data for collection1 to ver 0 
   [junit4]   2> 1474425 INFO  (coreLoadExecutor-1697-thread-1-processing-n:127.0.0.1:45490_le%2Fb) [n:127.0.0.1:45490_le%2Fb c:collection1   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 1474526 INFO  (OverseerStateUpdate-94154951802355717-127.0.0.1:55381_le%2Fb-n_0000000000) [n:127.0.0.1:55381_le%2Fb    ] o.a.s.c.o.ZkStateWriter going to update_collection /collections/collection1/state.json version: 0
   [junit4]   2> 1474527 INFO  (zkCallback-465-thread-1-processing-n:127.0.0.1:45490_le%2Fb) [n:127.0.0.1:45490_le%2Fb    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json for collection collection1 has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1474527 INFO  (zkCallback-465-thread-1-processing-n:127.0.0.1:45490_le%2Fb) [n:127.0.0.1:45490_le%2Fb    ] o.a.s.c.c.ZkStateReader Updating data for collection1 to ver 1 
   [junit4]   2> 1475425 INFO  (coreLoadExecutor-1697-thread-1-processing-n:127.0.0.1:45490_le%2Fb) [n:127.0.0.

[...truncated too long message...]

ns/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/apacheds-protocol-kerberos-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/apacheds-protocol-ldap-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/apacheds-protocol-shared-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/apacheds-xdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/api-all-1.0.0-M20.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/bcprov-jdk15-1.45.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/cglib-nodep-2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/commons-collections-3.2.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/commons-math3-3.4.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/easymock-3.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/ehcache-core-2.4.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/hadoop-common-2.6.0-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/hadoop-hdfs-2.6.0-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/hadoop-minikdc-2.6.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/jackson-annotations-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/jackson-databind-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/jcl-over-slf4j-1.7.7.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/jersey-core-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/jersey-server-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/jetty-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/jetty-util-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/mina-core-2.0.0-M5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/objenesis-1.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/presto-parser-0.108.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/slice-0.10.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/build/analysis/icu/lucene-analyzers-icu-6.0.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/contrib/solr-analysis-extras/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/contrib/analysis-extras/lib/icu4j-54.1.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-launcher.jar:/x1/jenkins/.ant/lib/ivy-2.3.0.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-antlr.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-regexp.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-resolver.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-net.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jmf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-oro.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-xalan2.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-logging.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jsch.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit4.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-javamail.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jai.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jdepend.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bsf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-swing.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-netrexx.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bcel.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-testutil.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-log4j.jar:/x1/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.8/lib/tools.jar:/x1/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.1.13.jar com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/temp/junit4-J1-20150712_041754_251.events @/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/temp/junit4-J1-20150712_041754_251.suites
   [junit4] ERROR: JVM J1 ended with an exception: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1504)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:133)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:964)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:961)
   [junit4] 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4] 	at java.lang.Thread.run(Thread.java:745)

BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:533: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:474: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:61: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/extra-targets.xml:39: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build.xml:230: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/common-build.xml:513: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/common-build.xml:1426: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/common-build.xml:981: At least one slave process threw an exception, first: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.

Total time: 456 minutes 7 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
No prior successful build to compare, so performing full copy of artifacts
ERROR: Failed to archive artifacts: **/*.events,heapdumps/**,**/hs_err_pid*,README.txt
java.io.IOException: Failed to extract /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/transfer of 4 files
	at hudson.FilePath.readFromTar(FilePath.java:2101)
	at hudson.FilePath.copyRecursiveTo(FilePath.java:2013)
	at jenkins.model.StandardArtifactManager.archive(StandardArtifactManager.java:61)
	at com.cloudbees.jenkins.plugins.jsync.archiver.JSyncArtifactManager.remoteSync(JSyncArtifactManager.java:107)
	at com.cloudbees.jenkins.plugins.jsync.archiver.JSyncArtifactManager.archive(JSyncArtifactManager.java:68)
	at hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:140)
	at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
	at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:756)
	at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:720)
	at hudson.model.Build$BuildExecution.post2(Build.java:182)
	at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:669)
	at hudson.model.Run.execute(Run.java:1731)
	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
	at hudson.model.ResourceController.execute(ResourceController.java:88)
	at hudson.model.Executor.run(Executor.java:232)
Caused by: java.io.IOException: unexpected EOF with 1024 bytes unread
	at hudson.org.apache.tools.tar.TarInputStream.read(TarInputStream.java:349)
	at java.io.FilterInputStream.read(FilterInputStream.java:107)
	at org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:1792)
	at org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:1769)
	at org.apache.commons.io.IOUtils.copy(IOUtils.java:1744)
	at hudson.util.IOUtils.copy(IOUtils.java:40)
	at hudson.FilePath.readFromTar(FilePath.java:2091)
	... 14 more
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-NightlyTests-trunk - Build # 736 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-trunk/736/

3 tests failed.
REGRESSION:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=5613, name=collection0, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=5613, name=collection0, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:33120/_g/b: Could not find collection : awholynewstresscollection_collection0_0
	at __randomizedtesting.SeedInfo.seed([BFEA587A71B28EC1]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:560)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:234)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:226)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:376)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:328)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1086)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:856)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:799)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:894)


REGRESSION:  org.apache.solr.cloud.hdfs.StressHdfsTest.test

Error Message:
Error from server at http://127.0.0.1:44959: Could not find collection : delete_data_dir

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:44959: Could not find collection : delete_data_dir
	at __randomizedtesting.SeedInfo.seed([BFEA587A71B28EC1:37BE67A0DF4EE339]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:560)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:234)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:226)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:376)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:328)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1086)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:856)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:799)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.hdfs.StressHdfsTest.createAndDeleteCollection(StressHdfsTest.java:204)
	at org.apache.solr.cloud.hdfs.StressHdfsTest.test(StressHdfsTest.java:98)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:483)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:960)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:935)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=48768, name=collection1, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=48768, name=collection1, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:35668: Could not find collection : awholynewstresscollection_collection1_0
	at __randomizedtesting.SeedInfo.seed([BFEA587A71B28EC1]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:560)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:234)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:226)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:376)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:328)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1086)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:856)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:799)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:894)




Build Log:
[...truncated 10175 lines...]
   [junit4] Suite: org.apache.solr.cloud.hdfs.StressHdfsTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.StressHdfsTest_BFEA587A71B28EC1-001/init-core-data-001
   [junit4]   2> 192860 INFO  (SUITE-StressHdfsTest-seed#[BFEA587A71B28EC1]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   1> Formatting using clusterid: testClusterID
   [junit4]   2> 192974 WARN  (SUITE-StressHdfsTest-seed#[BFEA587A71B28EC1]-worker) [    ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
   [junit4]   2> 193010 WARN  (SUITE-StressHdfsTest-seed#[BFEA587A71B28EC1]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 193021 INFO  (SUITE-StressHdfsTest-seed#[BFEA587A71B28EC1]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 193071 INFO  (SUITE-StressHdfsTest-seed#[BFEA587A71B28EC1]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/hdfs to ./temp/Jetty_localhost_50456_hdfs____9cmfpg/webapp
   [junit4]   2> 193322 INFO  (SUITE-StressHdfsTest-seed#[BFEA587A71B28EC1]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 193885 INFO  (SUITE-StressHdfsTest-seed#[BFEA587A71B28EC1]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:50456
   [junit4]   2> 194147 WARN  (SUITE-StressHdfsTest-seed#[BFEA587A71B28EC1]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 194149 INFO  (SUITE-StressHdfsTest-seed#[BFEA587A71B28EC1]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 194170 INFO  (SUITE-StressHdfsTest-seed#[BFEA587A71B28EC1]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_57190_datanode____.2v9beb/webapp
   [junit4]   2> 194381 INFO  (SUITE-StressHdfsTest-seed#[BFEA587A71B28EC1]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 195041 INFO  (SUITE-StressHdfsTest-seed#[BFEA587A71B28EC1]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:57190
   [junit4]   2> 195110 WARN  (SUITE-StressHdfsTest-seed#[BFEA587A71B28EC1]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 195113 INFO  (SUITE-StressHdfsTest-seed#[BFEA587A71B28EC1]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 195159 INFO  (SUITE-StressHdfsTest-seed#[BFEA587A71B28EC1]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_55258_datanode____xkq4v0/webapp
   [junit4]   2> 195376 INFO  (IPC Server handler 2 on 53676) [    ] BlockStateChange BLOCK* processReport: from storage DS-b6cf5356-6be7-4dca-a97d-c443b162d983 node DatanodeRegistration(127.0.0.1, datanodeUuid=7d9c57cd-6d94-489b-8e4a-ce21e481e40c, infoPort=57190, ipcPort=51099, storageInfo=lv=-56;cid=testClusterID;nsid=609059039;c=0), blocks: 0, hasStaleStorages: true, processing time: 1 msecs
   [junit4]   2> 195377 INFO  (IPC Server handler 2 on 53676) [    ] BlockStateChange BLOCK* processReport: from storage DS-66983b53-a017-4c2a-ae69-6b9e6f2f0bc5 node DatanodeRegistration(127.0.0.1, datanodeUuid=7d9c57cd-6d94-489b-8e4a-ce21e481e40c, infoPort=57190, ipcPort=51099, storageInfo=lv=-56;cid=testClusterID;nsid=609059039;c=0), blocks: 0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 195392 INFO  (SUITE-StressHdfsTest-seed#[BFEA587A71B28EC1]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 196049 INFO  (SUITE-StressHdfsTest-seed#[BFEA587A71B28EC1]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:55258
   [junit4]   2> 196229 INFO  (IPC Server handler 3 on 53676) [    ] BlockStateChange BLOCK* processReport: from storage DS-5cac1766-cd52-42c2-9aea-394027ad3bde node DatanodeRegistration(127.0.0.1, datanodeUuid=511acd6c-0a9c-4e7f-a093-64658d14c43f, infoPort=55258, ipcPort=33655, storageInfo=lv=-56;cid=testClusterID;nsid=609059039;c=0), blocks: 0, hasStaleStorages: true, processing time: 1 msecs
   [junit4]   2> 196230 INFO  (IPC Server handler 3 on 53676) [    ] BlockStateChange BLOCK* processReport: from storage DS-531e92bb-6e7e-4fa2-b8aa-264aa31d2725 node DatanodeRegistration(127.0.0.1, datanodeUuid=511acd6c-0a9c-4e7f-a093-64658d14c43f, infoPort=55258, ipcPort=33655, storageInfo=lv=-56;cid=testClusterID;nsid=609059039;c=0), blocks: 0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 196336 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 196336 INFO  (Thread-435) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 196336 INFO  (Thread-435) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 196436 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [    ] o.a.s.c.ZkTestServer start zk server on port:49245
   [junit4]   2> 196437 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 196459 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 196472 INFO  (zkCallback-37-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1731274e name:ZooKeeperConnection Watcher:127.0.0.1:49245 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 196472 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 196473 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 196473 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 196481 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 196489 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 196508 INFO  (zkCallback-38-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3c0b9476 name:ZooKeeperConnection Watcher:127.0.0.1:49245/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 196508 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 196509 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 196509 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 196525 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 196532 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 196534 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 196536 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 196537 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 196539 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 196540 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 196542 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 196542 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 196544 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 196544 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 196546 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 196546 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 196547 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 196548 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 196549 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 196549 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 196551 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 196551 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 196553 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 196553 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 196555 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 196555 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 196557 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 196557 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 197018 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.StressHdfsTest_BFEA587A71B28EC1-001/control-001/cores/collection1
   [junit4]   2> 197020 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [    ] o.e.j.s.Server jetty-9.2.11.v20150529
   [junit4]   2> 197031 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@6de5a21b{/,null,AVAILABLE}
   [junit4]   2> 197032 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [    ] o.e.j.s.ServerConnector Started ServerConnector@4f5dddd2{HTTP/1.1}{127.0.0.1:52482}
   [junit4]   2> 197032 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [    ] o.e.j.s.Server Started @200273ms
   [junit4]   2> 197032 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=hdfs://localhost:53676/hdfs__localhost_53676__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-trunk_solr_build_solr-core_test_J2_temp_solr.cloud.hdfs.StressHdfsTest_BFEA587A71B28EC1-001_tempDir-002_control_data, hostContext=/, hostPort=52482, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.StressHdfsTest_BFEA587A71B28EC1-001/control-001/cores}
   [junit4]   2> 197033 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@c387f44
   [junit4]   2> 197033 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.StressHdfsTest_BFEA587A71B28EC1-001/control-001/'
   [junit4]   2> 197051 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.StressHdfsTest_BFEA587A71B28EC1-001/control-001/solr.xml
   [junit4]   2> 197070 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.StressHdfsTest_BFEA587A71B28EC1-001/control-001/cores
   [junit4]   2> 197071 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [    ] o.a.s.c.CoreContainer New CoreContainer 503712654
   [junit4]   2> 197071 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.StressHdfsTest_BFEA587A71B28EC1-001/control-001/]
   [junit4]   2> 197071 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.StressHdfsTest_BFEA587A71B28EC1-001/control-001/lib
   [junit4]   2> 197074 WARN  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.StressHdfsTest_BFEA587A71B28EC1-001/control-001/lib).
   [junit4]   2> 197098 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 197103 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 197104 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 197104 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 197104 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:49245/solr
   [junit4]   2> 197104 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 197104 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 197136 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 197147 INFO  (zkCallback-40-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2f1242dc name:ZooKeeperConnection Watcher:127.0.0.1:49245 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 197147 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 197148 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 197176 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [n:127.0.0.1:52482_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 197201 INFO  (zkCallback-41-thread-1-processing-n:127.0.0.1:52482_) [n:127.0.0.1:52482_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@32d09f7e name:ZooKeeperConnection Watcher:127.0.0.1:49245/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 197202 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [n:127.0.0.1:52482_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 197203 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [n:127.0.0.1:52482_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 197212 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [n:127.0.0.1:52482_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 197214 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [n:127.0.0.1:52482_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 197217 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [n:127.0.0.1:52482_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 197219 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [n:127.0.0.1:52482_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 197221 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [n:127.0.0.1:52482_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 197223 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [n:127.0.0.1:52482_    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 197224 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [n:127.0.0.1:52482_    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 197225 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [n:127.0.0.1:52482_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:52482_
   [junit4]   2> 197226 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [n:127.0.0.1:52482_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:52482_
   [junit4]   2> 197238 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [n:127.0.0.1:52482_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 197239 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [n:127.0.0.1:52482_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 197241 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [n:127.0.0.1:52482_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 197241 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [n:127.0.0.1:52482_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94148556074188803-127.0.0.1:52482_-n_0000000000
   [junit4]   2> 197242 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [n:127.0.0.1:52482_    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:52482_
   [junit4]   2> 197242 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [n:127.0.0.1:52482_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 197243 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [n:127.0.0.1:52482_    ] o.a.s.c.Overseer Overseer (id=94148556074188803-127.0.0.1:52482_-n_0000000000) starting
   [junit4]   2> 197245 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [n:127.0.0.1:52482_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 197259 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [n:127.0.0.1:52482_    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 197265 INFO  (OverseerStateUpdate-94148556074188803-127.0.0.1:52482_-n_0000000000) [n:127.0.0.1:52482_    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 197269 INFO  (OverseerCollectionProcessor-94148556074188803-127.0.0.1:52482_-n_0000000000) [n:127.0.0.1:52482_    ] o.a.s.c.OverseerCollectionProcessor Process current queue of collection creations
   [junit4]   2> 197278 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [n:127.0.0.1:52482_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 197281 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [n:127.0.0.1:52482_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 197281 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [n:127.0.0.1:52482_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 197282 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [n:127.0.0.1:52482_    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.StressHdfsTest_BFEA587A71B28EC1-001/control-001/cores
   [junit4]   2> 197283 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [n:127.0.0.1:52482_    ] o.a.s.c.SolrCore Created CoreDescriptor: {name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, loadOnStartup=true, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.StressHdfsTest_BFEA587A71B28EC1-001/control-001/cores/collection1, collection=control_collection, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.StressHdfsTest_BFEA587A71B28EC1-001/control-001/cores/collection1/, coreNodeName=, dataDir=data/, shard=}
   [junit4]   2> 197283 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [n:127.0.0.1:52482_    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.StressHdfsTest_BFEA587A71B28EC1-001/control-001/cores/collection1/
   [junit4]   2> 197283 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [n:127.0.0.1:52482_    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 197291 INFO  (coreLoadExecutor-231-thread-1-processing-n:127.0.0.1:52482_) [n:127.0.0.1:52482_ c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 197291 INFO  (coreLoadExecutor-231-thread-1-processing-n:127.0.0.1:52482_) [n:127.0.0.1:52482_ c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 197305 INFO  (zkCallback-41-thread-1-processing-n:127.0.0.1:52482_) [n:127.0.0.1:52482_    ] o.a.s.c.DistributedQueue NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 197305 INFO  (coreLoadExecutor-231-thread-1-processing-n:127.0.0.1:52482_) [n:127.0.0.1:52482_ c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 197306 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [n:127.0.0.1:52482_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2
   [junit4]   2> 197306 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [n:127.0.0.1:52482_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 197308 INFO  (OverseerStateUpdate-94148556074188803-127.0.0.1:52482_-n_0000000000) [n:127.0.0.1:52482_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:52482",
   [junit4]   2>   "node_name":"127.0.0.1:52482_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 197309 INFO  (OverseerStateUpdate-94148556074188803-127.0.0.1:52482_-n_0000000000) [n:127.0.0.1:52482_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:52482",
   [junit4]   2>   "node_name":"127.0.0.1:52482_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 197309 INFO  (OverseerStateUpdate-94148556074188803-127.0.0.1:52482_-n_0000000000) [n:127.0.0.1:52482_    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 197310 INFO  (OverseerStateUpdate-94148556074188803-127.0.0.1:52482_-n_0000000000) [n:127.0.0.1:52482_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 197314 INFO  (zkCallback-41-thread-1-processing-n:127.0.0.1:52482_) [n:127.0.0.1:52482_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 197316 INFO  (zkCallback-41-thread-1-processing-n:127.0.0.1:52482_) [n:127.0.0.1:52482_    ] o.a.s.c.c.ZkStateReader Updated cluster state version to 1
   [junit4]   2> 198308 INFO  (coreLoadExecutor-231-thread-1-processing-n:127.0.0.1:52482_) [n:127.0.0.1:52482_ c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 198308 INFO  (coreLoadExecutor-231-thread-1-processing-n:127.0.0.1:52482_) [n:127.0.0.1:52482_ c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 198310 INFO  (coreLoadExecutor-231-thread-1-processing-n:127.0.0.1:52482_) [n:127.0.0.1:52482_ c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 198310 INFO  (coreLoadExecutor-231-thread-1-processing-n:127.0.0.1:52482_) [n:127.0.0.1:52482_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 198312 INFO  (coreLoadExecutor-231-thread-1-processing-n:127.0.0.1:52482_) [n:127.0.0.1:52482_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 198312 INFO  (coreLoadExecutor-231-thread-1-processing-n:127.0.0.1:52482_) [n:127.0.0.1:52482_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.StressHdfsTest_BFEA587A71B28EC1-001/control-001/cores/collection1/'
   [junit4]   2> 198336 INFO  (coreLoadExecutor-231-thread-1-processing-n:127.0.0.1:52482_) [n:127.0.0.1:52482_ c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 198360 INFO  (coreLoadExecutor-231-thread-1-processing-n:127.0.0.1:52482_) [n:127.0.0.1:52482_ c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 198376 INFO  (coreLoadExecutor-231-thread-1-processing-n:127.0.0.1:52482_) [n:127.0.0.1:52482_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.0.0
   [junit4]   2> 198398 INFO  (coreLoadExecutor-231-thread-1-processing-n:127.0.0.1:52482_) [n:127.0.0.1:52482_ c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 198399 INFO  (coreLoadExecutor-231-thread-1-processing-n:127.0.0.1:52482_) [n:127.0.0.1:52482_ c:control_collection   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 198404 INFO  (coreLoadExecutor-231-thread-1-processing-n:127.0.0.1:52482_) [n:127.0.0.1:52482_ c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 198527 INFO  (coreLoadExecutor-231-thread-1-processing-n:127.0.0.1:52482_) [n:127.0.0.1:52482_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 198533 INFO  (coreLoadExecutor-231-thread-1-processing-n:127.0.0.1:52482_) [n:127.0.0.1:52482_ c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 198537 INFO  (coreLoadExecutor-231-thread-1-processing-n:127.0.0.1:52482_) [n:127.0.0.1:52482_ c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 198562 INFO  (coreLoadExecutor-231-thread-1-processing-n:127.0.0.1:52482_) [n:127.0.0.1:52482_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 198567 INFO  (coreLoadExecutor-231-thread-1-processing-n:127.0.0.1:52482_) [n:127.0.0.1:52482_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 198571 INFO  (coreLoadExecutor-231-thread-1-processing-n:127.0.0.1:52482_) [n:127.0.0.1:52482_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 198573 WARN  (coreLoadExecutor-231-thread-1-processing-n:127.0.0.1:52482_) [n:127.0.0.1:52482_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 198574 WARN  (coreLoadExecutor-231-thread-1-processing-n:127.0.0.1:52482_) [n:127.0.0.1:52482_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 198574 INFO  (coreLoadExecutor-231-thread-1-processing-n:127.0.0.1:52482_) [n:127.0.0.1:52482_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 198575 WARN  (coreLoadExecutor-231-thread-1-processing-n:127.0.0.1:52482_) [n:127.0.0.1:52482_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 198575 WARN  (coreLoadExecutor-231-thread-1-processing-n:127.0.0.1:52482_) [n:127.0.0.1:52482_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 198575 INFO  (coreLoadExecutor-231-thread-1-processing-n:127.0.0.1:52482_) [n:127.0.0.1:52482_ c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 198575 INFO  (coreLoadExecutor-231-thread-1-processing-n:127.0.0.1:52482_) [n:127.0.0.1:52482_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.HdfsDirectoryFactory
   [junit4]   2> 198586 INFO  (coreLoadExecutor-231-thread-1-processing-n:127.0.0.1:52482_) [n:127.0.0.1:52482_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:53676/solr_hdfs_home
   [junit4]   2> 198586 INFO  (coreLoadExecutor-231-thread-1-processing-n:127.0.0.1:52482_) [n:127.0.0.1:52482_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
   [junit4]   2> 198587 INFO  (coreLoadExecutor-231-thread-1-processing-n:127.0.0.1:52482_) [n:127.0.0.1:52482_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.StressHdfsTest_BFEA587A71B28EC1-001/control-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 198587 INFO  (coreLoadExecutor-231-thread-1-processing-n:127.0.0.1:52482_) [n:127.0.0.1:52482_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@21713abe
   [junit4]   2> 198587 INFO  (coreLoadExecutor-231-thread-1-processing-n:127.0.0.1:52482_) [n:127.0.0.1:52482_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:53676/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 198634 WARN  (coreLoadExecutor-231-thread-1-processing-n:127.0.0.1:52482_) [n:127.0.0.1:52482_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.h.HdfsDirectory The NameNode is in SafeMode - Solr will wait 5 seconds and try again.
   [junit4]   2> 203638 WARN  (coreLoadExecutor-231-thread-1-processing-n:127.0.0.1:52482_) [n:127.0.0.1:52482_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.h.HdfsDirectory The NameNode is in SafeMode - Solr will wait 5 seconds and try again.
   [junit4]   2> 208658 INFO  (coreLoadExecutor-231-thread-1-processing-n:127.0.0.1:52482_) [n:127.0.0.1:52482_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.h.HdfsLocalityReporter Registering direcotry hdfs://localhost:53676/solr_hdfs_home/control_collection/core_node1/data for locality metrics.
   [junit4]   2> 208659 INFO  (coreLoadExecutor-231-thread-1-processing-n:127.0.0.1:52482_) [n:127.0.0.1:52482_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:53676/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 208660 INFO  (coreLoadExecutor-231-thread-1-processing-n:127.0.0.1:52482_) [n:127.0.0.1:52482_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=hdfs://localhost:53676/solr_hdfs_home/control_collection/core_node1/data/index/
   [junit4]   2> 208708 WARN  (coreLoadExecutor-231-thread-1-processing-n:127.0.0.1:52482_) [n:127.0.0.1:52482_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory 'hdfs:/localhost:53676/solr_hdfs_home/control_collection/core_node1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 208708 INFO  (coreLoadExecutor-231-thread-1-processing-n:127.0.0.1:52482_) [n:127.0.0.1:52482_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:53676/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 208732 INFO  (coreLoadExecutor-231-thread-1-processing-n:127.0.0.1:52482_) [n:127.0.0.1:52482_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
   [junit4]   2> 208733 INFO  (coreLoadExecutor-231-thread-1-processing-n:127.0.0.1:52482_) [n:127.0.0.1:52482_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
   [junit4]   2> 208733 INFO  (coreLoadExecutor-231-thread-1-processing-n:127.0.0.1:52482_) [n:127.0.0.1:52482_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Creating new global HDFS BlockCache
   [junit4]   2> 210572 INFO  (coreLoadExecutor-231-thread-1-processing-n:127.0.0.1:52482_) [n:127.0.0.1:52482_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
   [junit4]   2> 210572 INFO  (coreLoadExecutor-231-thread-1-processing-n:127.0.0.1:52482_) [n:127.0.0.1:52482_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.h.HdfsLocalityReporter Registering direcotry hdfs://localhost:53676/solr_hdfs_home/control_collection/core_node1/data/index for locality metrics.
   [junit4]   2> 210572 INFO  (coreLoadExecutor-231-thread-1-processing-n:127.0.0.1:52482_) [n:127.0.0.1:52482_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:53676/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 210573 INFO  (coreLoadExecutor-231-thread-1-processing-n:127.0.0.1:52482_) [n:127.0.0.1:52482_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=18, maxMergeAtOnceExplicit=26, maxMergedSegmentMB=32.4111328125, floorSegmentMB=0.4814453125, forceMergeDeletesPctAllowed=16.39665767607123, segmentsPerTier=13.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.2503651975721729
   [junit4]   2> 210636 INFO  (IPC Server handler 9 on 53676) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:50350 is added to blk_1073741825_1001{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-5cac1766-cd52-42c2-9aea-394027ad3bde:NORMAL:127.0.0.1:60114|RBW], ReplicaUnderConstruction[[DISK]DS-b6cf5356-6be7-4dca-a97d-c443b162d983:NORMAL:127.0.0.1:50350|FINALIZED]]} size 0
   [junit4]   2> 210646 INFO  (IPC Server handler 5 on 53676) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:60114 is added to blk_1073741825_1001{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-5cac1766-cd52-42c2-9aea-394027ad3bde:NORMAL:127.0.0.1:60114|RBW], ReplicaUnderConstruction[[DISK]DS-b6cf5356-6be7-4dca-a97d-c443b162d983:NORMAL:127.0.0.1:50350|FINALIZED]]} size 0
   [junit4]   2> 210665 INFO  (coreLoadExecutor-231-thread-1-processing-n:127.0.0.1:52482_) [n:127.0.0.1:52482_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(BlockDirectory(HdfsDirectory@b8659af2 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@618230bb); maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 210666 INFO  (coreLoadExecutor-231-thread-1-processing-n:127.0.0.1:52482_) [n:127.0.0.1:52482_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 210686 INFO  (coreLoadExecutor-231-thread-1-processing-n:127.0.0.1:52482_) [n:127.0.0.1:52482_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 210687 INFO  (coreLoadExecutor-231-thread-1-processing-n:127.0.0.1:52482_) [n:127.0.0.1:52482_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 210687 INFO  (coreLoadExecutor-231-thread-1-processing-n:127.0.0.1:52482_) [n:127.0.0.1:52482_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 210687 INFO  (coreLoadExecutor-231-thread-1-processing-n:127.0.0.1:52482_) [n:127.0.0.1:52482_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 210687 INFO  (coreLoadExecutor-231-thread-1-processing-n:127.0.0.1:52482_) [n:127.0.0.1:52482_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 210687 INFO  (coreLoadExecutor-231-thread-1-processing-n:127.0.0.1:52482_) [n:127.0.0.1:52482_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 210688 INFO  (coreLoadExecutor-231-thread-1-processing-n:127.0.0.1:52482_) [n:127.0.0.1:52482_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 210688 INFO  (coreLoadExecutor-231-thread-1-processing-n:127.0.0.1:52482_) [n:127.0.0.1:52482_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 210688 INFO  (coreLoadExecutor-231-thread-1-processing-n:127.0.0.1:52482_) [n:127.0.0.1:52482_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 210689 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:52482_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in hdfs://localhost:53676/solr_hdfs_home/control_collection/core_node1/data/
   [junit4]   2> 210690 INFO  (coreLoadExecutor-231-thread-1-processing-n:127.0.0.1:52482_) [n:127.0.0.1:52482_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 210691 INFO  (coreLoadExecutor-231-thread-1-processing-n:127.0.0.1:52482_) [n:127.0.0.1:52482_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 210692 INFO  (coreLoadExecutor-231-thread-1-processing-n:127.0.0.1:52482_) [n:127.0.0.1:52482_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 210693 INFO  (coreLoadExecutor-231-thread-1-processing-n:127.0.0.1:52482_) [n:127.0.0.1:52482_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 210696 INFO  (coreLoadExecutor-231-thread-1-processing-n:127.0.0.1:52482_) [n:127.0.0.1:52482_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/mbeans,standard,/update/csv,/update/json/docs,/admin/luke,/admin/segments,/get,/admin/system,/replication,/admin/properties,/config,/schema,/admin/plugins,/admin/logging,/update/json,/admin/threads,/admin/ping,/update,/admin/file
   [junit4]   2> 210697 INFO  (coreLoadExecutor-231-thread-1-processing-n:127.0.0.1:52482_) [n:127.0.0.1:52482_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 210703 INFO  (coreLoadExecutor-231-thread-1-processing-n:127.0.0.1:52482_) [n:127.0.0.1:52482_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
   [junit4]   2> 210703 INFO  (coreLoadExecutor-231-thread-1-processing-n:127.0.0.1:52482_) [n:127.0.0.1:52482_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing HdfsUpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 tlogDfsReplication=2
   [junit4]   2> 210730 INFO  (coreLoadExecutor-231-thread-1-processing-n:127.0.0.1:52482_) [n:127.0.0.1:52482_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 210730 INFO  (coreLoadExecutor-231-thread-1-processing-n:127.0.0.1:52482_) [n:127.0.0.1:52482_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 210734 INFO  (coreLoadExecutor-231-thread-1-processing-n:127.0.0.1:52482_) [n:127.0.0.1:52482_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@578cb0b4
   [junit4]   2> 210797 INFO  (coreLoadExecutor-231-thread-1-processing-n:127.0.0.1:52482_) [n:127.0.0.1:52482_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(BlockDirectory(HdfsDirectory@b8659af2 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@618230bb); maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 210797 INFO  (coreLoadExecutor-231-thread-1-processing-n:127.0.0.1:52482_) [n:127.0.0.1:52482_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 210797 INFO  (coreLoadExecutor-231-thread-1-processing-n:127.0.0.1:52482_) [n:127.0.0.1:52482_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@111ae5b9[collection1] main
   [junit4]   2> 210798 INFO  (coreLoadExecutor-231-thread-1-processing-n:127.0.0.1:52482_) [n:127.0.0.1:52482_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 210799 INFO  (coreLoadExecutor-231-thread-1-processing-n:127.0.0.1:52482_) [n:127.0.0.1:52482_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 210799 INFO  (coreLoadExecutor-231-thread-1-processing-n:127.0.0.1:52482_) [n:127.0.0.1:52482_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 210800 INFO  (coreLoadExecutor-231-thread-1-processing-n:127.0.0.1:52482_) [n:127.0.0.1:52482_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 210800 INFO  (coreLoadExecutor-231-thread-1-processing-n:127.0.0.1:52482_) [n:127.0.0.1:52482_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 210800 INFO  (coreLoadExecutor-231-thread-1-processing-n:127.0.0.1:52482_) [n:127.0.0.1:52482_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 210800 INFO  (coreLoadExecutor-231-thread-1-processing-n:127.0.0.1:52482_) [n:127.0.0.1:52482_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 210801 INFO  (coreLoadExecutor-231-thread-1-processing-n:127.0.0.1:52482_) [n:127.0.0.1:52482_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 210801 INFO  (coreLoadExecutor-231-thread-1-processing-n:127.0.0.1:52482_) [n:127.0.0.1:52482_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 210801 INFO  (coreLoadExecutor-231-thread-1-processing-n:127.0.0.1:52482_) [n:127.0.0.1:52482_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 210801 INFO  (searcherExecutor-232-thread-1-processing-n:127.0.0.1:52482_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:52482_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@111ae5b9[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 210815 INFO  (coreLoadExecutor-231-thread-1-processing-n:127.0.0.1:52482_) [n:127.0.0.1:52482_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 210816 INFO  (coreLoadExecutor-231-thread-1-processing-n:127.0.0.1:52482_) [n:127.0.0.1:52482_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 256 version buckets from index
   [junit4]   2> 210816 INFO  (coreLoadExecutor-231-thread-1-processing-n:127.0.0.1:52482_) [n:127.0.0.1:52482_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 210816 INFO  (coreLoadExecutor-231-thread-1-processing-n:127.0.0.1:52482_) [n:127.0.0.1:52482_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1506376912367255552
   [junit4]   2> 210816 INFO  (coreLoadExecutor-231-thread-1-processing-n:127.0.0.1:52482_) [n:127.0.0.1:52482_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 0 ms to seed version buckets with highest version 1506376912367255552
   [junit4]   2> 210816 INFO  (coreLoadExecutor-231-thread-1-processing-n:127.0.0.1:52482_) [n:127.0.0.1:52482_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 210817 INFO  (coreLoadExecutor-231-thread-1-processing-n:127.0.0.1:52482_) [n:127.0.0.1:52482_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 210817 INFO  (coreZkRegister-225-thread-1-processing-n:127.0.0.1:52482_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:52482_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:52482 collection:control_collection shard:shard1
   [junit4]   2> 210818 INFO  (coreZkRegister-225-thread-1-processing-n:127.0.0.1:52482_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:52482_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 210823 INFO  (coreZkRegister-225-thread-1-processing-n:127.0.0.1:52482_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:52482_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94148556074188803-core_node1-n_0000000000
   [junit4]   2> 210824 INFO  (coreZkRegister-225-thread-1-processing-n:127.0.0.1:52482_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:52482_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 210825 INFO  (zkCallback-41-thread-1-processing-n:127.0.0.1:52482_) [n:127.0.0.1:52482_    ] o.a.s.c.DistributedQueue NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 210826 INFO  (OverseerStateUpdate-94148556074188803-127.0.0.1:52482_-n_0000000000) [n:127.0.0.1:52482_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 210827 INFO  (coreZkRegister-225-thread-1-processing-n:127.0.0.1:52482_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:52482_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 210827 INFO  (coreZkRegister-225-thread-1-processing-n:127.0.0.1:52482_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:52482_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 210828 INFO  (zkCallback-41-thread-1-processing-n:127.0.0.1:52482_) [n:127.0.0.1:52482_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 210829 INFO  (coreZkRegister-225-thread-1-processing-n:127.0.0.1:52482_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:52482_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:52482/collection1/
   [junit4]   2> 210830 INFO  (coreZkRegister-225-thread-1-processing-n:127.0.0.1:52482_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:52482_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 210830 INFO  (coreZkRegister-225-thread-1-processing-n:127.0.0.1:52482_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:52482_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:52482/collection1/ has no replicas
   [junit4]   2> 210830 INFO  (coreZkRegister-225-thread-1-processing-n:127.0.0.1:52482_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:52482_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:52482/collection1/ shard1
   [junit4]   2> 210830 INFO  (coreZkRegister-225-thread-1-processing-n:127.0.0.1:52482_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:52482_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 210832 INFO  (zkCallback-41-thread-1-processing-n:127.0.0.1:52482_) [n:127.0.0.1:52482_    ] o.a.s.c.c.ZkStateReader Updated cluster state version to 2
   [junit4]   2> 210836 INFO  (zkCallback-41-thread-1-processing-n:127.0.0.1:52482_) [n:127.0.0.1:52482_    ] o.a.s.c.DistributedQueue NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 210839 INFO  (OverseerStateUpdate-94148556074188803-127.0.0.1:52482_-n_0000000000) [n:127.0.0.1:52482_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:52482",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 2
   [junit4]   2> 210943 INFO  (zkCallback-41-thread-1-processing-n:127.0.0.1:52482_) [n:127.0.0.1:52482_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 210945 INFO  (zkCallback-41-thread-1-processing-n:127.0.0.1:52482_) [n:127.0.0.1:52482_    ] o.a.s.c.c.ZkStateReader Updated cluster state version to 3
   [junit4]   2> 210993 INFO  (coreZkRegister-225-thread-1-processing-n:127.0.0.1:52482_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:52482_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:52482/collection1/ and leader is http://127.0.0.1:52482/collection1/
   [junit4]   2> 210993 INFO  (coreZkRegister-225-thread-1-processing-n:127.0.0.1:52482_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:52482_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:52482
   [junit4]   2> 210993 INFO  (coreZkRegister-225-thread-1-processing-n:127.0.0.1:52482_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:52482_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 210993 INFO  (coreZkRegister-225-thread-1-processing-n:127.0.0.1:52482_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:52482_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 210993 INFO  (coreZkRegister-225-thread-1-processing-n:127.0.0.1:52482_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:52482_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 210995 INFO  (zkCallback-41-thread-1-processing-n:127.0.0.1:52482_) [n:127.0.0.1:52482_    ] o.a.s.c.DistributedQueue NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 210996 INFO  (OverseerStateUpdate-94148556074188803-127.0.0.1:52482_-n_0000000000) [n:127.0.0.1:52482_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:52482",
   [junit4]   2>   "node_name":"127.0.0.1:52482_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"} current state version: 3
   [junit4]   2> 210996 INFO  (OverseerStateUpdate-94148556074188803-127.0.0.1:52482_-n_0000000000) [n:127.0.0.1:52482_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:52482",
   [junit4]   2>   "node_name":"127.0.0.1:52482_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 211100 INFO  (zkCallback-41-thread-1-processing-n:127.0.0.1:52482_) [n:127.0.0.1:52482_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 211101 INFO  (zkCallback-41-thread-1-processing-n:127.0.0.1:52482_) [n:127.0.0.1:52482_    ] o.a.s.c.c.ZkStateReader Updated cluster state version to 4
   [junit4]   2> 211312 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 211312 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 211314 INFO  (zkCallback-43-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@78399a3d name:ZooKeeperConnection Watcher:127.0.0.1:49245/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 211314 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 211315 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 211315 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 211318 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 211736 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.StressHdfsTest_BFEA587A71B28EC1-001/shard-1-001/cores/collection1
   [junit4]   2> 211738 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.StressHdfsTest_BFEA587A71B28EC1-001/shard-1-001
   [junit4]   2> 211739 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [    ] o.e.j.s.Server jetty-9.2.11.v20150529
   [junit4]   2> 211794 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5e11f5ae{/,null,AVAILABLE}
   [junit4]   2> 211795 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [    ] o.e.j.s.ServerConnector Started ServerConnector@6d1f42b5{HTTP/1.1}{127.0.0.1:58721}
   [junit4]   2> 211795 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [    ] o.e.j.s.Server Started @215036ms
   [junit4]   2> 211795 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=hdfs://localhost:53676/hdfs__localhost_53676__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-trunk_solr_build_solr-core_test_J2_temp_solr.cloud.hdfs.StressHdfsTest_BFEA587A71B28EC1-001_tempDir-002_jetty1, solrconfig=solrconfig.xml, hostContext=/, hostPort=58721, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.StressHdfsTest_BFEA587A71B28EC1-001/shard-1-001/cores}
   [junit4]   2> 211796 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@c387f44
   [junit4]   2> 211796 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.StressHdfsTest_BFEA587A71B28EC1-001/shard-1-001/'
   [junit4]   2> 211820 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.StressHdfsTest_BFEA587A71B28EC1-001/shard-1-001/solr.xml
   [junit4]   2> 211825 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.StressHdfsTest_BFEA587A71B28EC1-001/shard-1-001/cores
   [junit4]   2> 211834 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [    ] o.a.s.c.CoreContainer New CoreContainer 2039297881
   [junit4]   2> 211834 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.StressHdfsTest_BFEA587A71B28EC1-001/shard-1-001/]
   [junit4]   2> 211834 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.StressHdfsTest_BFEA587A71B28EC1-001/shard-1-001/lib
   [junit4]   2> 211835 WARN  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.StressHdfsTest_BFEA587A71B28EC1-001/shard-1-001/lib).
   [junit4]   2> 211852 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 211858 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 211858 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 211858 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 211859 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:49245/solr
   [junit4]   2> 211859 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 211859 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 211889 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 211905 INFO  (zkCallback-45-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3cd4d8fe name:ZooKeeperConnection Watcher:127.0.0.1:49245 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 211906 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 211907 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 211910 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [n:127.0.0.1:58721_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 211946 INFO  (zkCallback-46-thread-1-processing-n:127.0.0.1:58721_) [n:127.0.0.1:58721_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2b956baa name:ZooKeeperConnection Watcher:127.0.0.1:49245/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 211947 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [n:127.0.0.1:58721_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 211960 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [n:127.0.0.1:58721_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 212968 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [n:127.0.0.1:58721_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:58721_
   [junit4]   2> 212969 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [n:127.0.0.1:58721_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:58721_
   [junit4]   2> 212975 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [n:127.0.0.1:58721_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 212976 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [n:127.0.0.1:58721_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94148556074188806-127.0.0.1:58721_-n_0000000001
   [junit4]   2> 212977 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [n:127.0.0.1:58721_    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/94148556074188803-127.0.0.1:52482_-n_0000000000 to know if I could be the leader
   [junit4]   2> 212977 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [n:127.0.0.1:58721_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 212978 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [n:127.0.0.1:58721_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 212978 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [n:127.0.0.1:58721_    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.StressHdfsTest_BFEA587A71B28EC1-001/shard-1-001/cores
   [junit4]   2> 212979 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [n:127.0.0.1:58721_    ] o.a.s.c.SolrCore Created CoreDescriptor: {name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, loadOnStartup=true, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.StressHdfsTest_BFEA587A71B28EC1-001/shard-1-001/cores/collection1, collection=collection1, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.StressHdfsTest_BFEA587A71B28EC1-001/shard-1-001/cores/collection1/, coreNodeName=, dataDir=data/, shard=}
   [junit4]   2> 212979 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [n:127.0.0.1:58721_    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.StressHdfsTest_BFEA587A71B28EC1-001/shard-1-001/cores/collection1/
   [junit4]   2> 212980 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [n:127.0.0.1:58721_    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 212980 INFO  (coreLoadExecutor-242-thread-1-processing-n:127.0.0.1:58721_) [n:127.0.0.1:58721_ c:collection1   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 212980 INFO  (coreLoadExecutor-242-thread-1-processing-n:127.0.0.1:58721_) [n:127.0.0.1:58721_ c:collection1   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 212981 INFO  (coreLoadExecutor-242-thread-1-processing-n:127.0.0.1:58721_) [n:127.0.0.1:58721_ c:collection1   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 212981 INFO  (zkCallback-41-thread-1-processing-n:127.0.0.1:52482_) [n:127.0.0.1:52482_    ] o.a.s.c.DistributedQueue NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 212986 INFO  (OverseerStateUpdate-94148556074188803-127.0.0.1:52482_-n_0000000000) [n:127.0.0.1:52482_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:58721",
   [junit4]   2>   "node_name":"127.0.0.1:58721_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "operation":"state"} current state version: 4
   [junit4]   2> 212986 INFO  (OverseerStateUpdate-94148556074188803-127.0.0.1:52482_-n_0000000000) [n:127.0.0.1:52482_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:58721",
   [junit4]   2>   "node_name":"127.0.0.1:58721_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 212986 INFO  (OverseerStateUpdate-94148556074188803-127.0.0.1:52482_-n_0000000000) [n:127.0.0.1:52482_    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 212987 INFO  (OverseerStateUpdate-94148556074188803-127.0.0.1:52482_-n_0000000000) [n:127.0.0.1:52482_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 212988 INFO  (zkCallback-41-thread-1-processing-n:127.0.0.1:52482_) [n:127.0.0.1:52482_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 212988 INFO  (zkCallback-46-thread-1-processing-n:127.0.0.1:58721_) [n:127.0.0.1:58721_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 212988 INFO  (zkCallback-43-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 212991 INFO  (zkCallback-46-thread-1-processing-n:127.0.0.1:58721_) [n:127.0.0.1:58721_    ] o.a.s.c.c.ZkStateReader Updated cluster state version to 5
   [junit4]   2> 212991 INFO  (zkCallback-41-thread-1-processing-n:127.0.0.1:52482_) [n:127.0.0.1:52482_    ] o.a.s.c.c.ZkStateReader Updated cluster state version to 5
   [junit4]   2> 212991 INFO  (zkCallback-43-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated cluster state version to 5
   [junit4]   2> 213009 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [n:127.0.0.1:58721_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2
   [junit4]   2> 213010 INFO  (TEST-StressHdfsTest.test-seed#[BFEA587A71B28EC1]) [n:127.0.0.1:58721_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 213982 INFO  (coreLoadExecutor-242-thread-1-processing-n:127.0.0.1:58721_) [n:127.0.0.1:58721_ c:collection1   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 213982 INFO  (coreLoadExecutor-242-thread-1-processing-n:127.0.0.1:58721_) [n:127.0.0.1:58721_ c:collection1   x:collection1] o.a.s.c.ZkController Check for collection zkNode:collection1
   [junit4]   2> 213983 INFO  (coreLoadExecutor-242-thread-1-processing-n:127.0.0.1:58721_) [n:127.0.0.1:58721_ c:collection1   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 213983 INFO  (coreLoadExecutor-242-thread-1-processing-n:127.0.0.1:58721_) [n:127.0.0.1:58721_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/collection1
   [junit4]   2> 213984 INFO  (coreLoadExecutor-242-thread-1-processing-n:127.0.0.1:58721_) [n:127.0.0.1:58721_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 213984 INFO  (coreLoadExecutor-242-thread-1-processing-n:127.0.0.1:58721_) [n:127.0.0.1:58721_ c:collection1   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.StressHdfsTest_BFEA587A71B28EC1-001/shard-1-001/cores/collection1/'
   [junit4]   2> 213997 INFO  (coreLoadExecutor-242-thread-1-processing-n:127.0.0.1:58721_) [n:127.0.0.1:58721_ c:collection1   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 214002 INFO  (coreLoadExecutor-242-thread-1-processing-n:127.0.0.1:58721_) [n:127.0.0.1:58721_ c:collection1   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 214006 INFO  (coreLoadExecutor-242-thread-1-processing-n:127.0.0.1:58721_) [n:127.0.0.1:58721_ c:collection1   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.0.0
   [junit4]   2> 214029 INFO  (coreLoadExecutor-242-thread-1-processing-n:127.0.0.1:58721_) [n:127.0.0.1:58721_ c:collection1   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 214031 INFO  (coreLoadExecutor-242-thread-1-processing-n:127.0.0.1:58721_) [n:127.0.0.1:58721_ c:collection1   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 214039 INFO  (coreLoadExecutor-242-thread-1-processing-n:127.0.0.1:58721_) [n:127.0.0.1:58721_ c:collection1   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 214154 INFO  (coreLoadExecutor-242-thread-1-processing-n:127.0.0.1:58721_) [n:127.0.0.1:58721_ c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 214162 INFO  (coreLoadExecutor-242-thread-1-processing-n:127.0.0.1:58721_) [n:127.0.0.1:58721_ c:collection1   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 214163 INFO  (coreLoadExecutor-242-thread-1-processing-n:127.0.0.1:58721_) [n:127.0.0.1:58721_ c:collection1   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 214174 INFO  (coreLoadExecutor-242-thread-1-processing-n:127.0.0.1:58721_) [n:127.0.0.1:58721_ c:collection1   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 214184 INFO  (coreLoadExecutor-242-thread-1-processing-n:127.0.0.1:58721_) [n:127.0.0.1:58721_ c:collection1   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 214190 INFO  (coreLoadExecutor-242-thread-1-processing-n:127.0.0.1:58721_) [n:127.0.0.1:58721_ c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading 

[...truncated too long message...]

p.metrics2.lib.MethodMetric$2.snapshot(MethodMetric.java:111)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.MethodMetric.snapshot(MethodMetric.java:144)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.MetricsRegistry.snapshot(MetricsRegistry.java:387)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.MetricsSourceBuilder$1.getMetrics(MetricsSourceBuilder.java:79)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.getMetrics(MetricsSourceAdapter.java:195)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.updateJmxCache(MetricsSourceAdapter.java:172)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.getMBeanInfo(MetricsSourceAdapter.java:151)
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getClassName(DefaultMBeanServerInterceptor.java:1804)
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.safeGetClassName(DefaultMBeanServerInterceptor.java:1595)
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.checkMBeanPermission(DefaultMBeanServerInterceptor.java:1813)
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.exclusiveUnregisterMBean(DefaultMBeanServerInterceptor.java:430)
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.unregisterMBean(DefaultMBeanServerInterceptor.java:415)
   [junit4]   2> 	at com.sun.jmx.mbeanserver.JmxMBeanServer.unregisterMBean(JmxMBeanServer.java:546)
   [junit4]   2> 	at org.apache.hadoop.metrics2.util.MBeans.unregister(MBeans.java:81)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.stopMBeans(MetricsSourceAdapter.java:227)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.stop(MetricsSourceAdapter.java:212)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.stopSources(MetricsSystemImpl.java:461)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.stop(MetricsSystemImpl.java:212)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.shutdown(MetricsSystemImpl.java:592)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.shutdownInstance(DefaultMetricsSystem.java:72)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.shutdown(DefaultMetricsSystem.java:68)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.metrics.NameNodeMetrics.shutdown(NameNodeMetrics.java:145)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.NameNode.stop(NameNode.java:822)
   [junit4]   2> 	at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1720)
   [junit4]   2> 	at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1699)
   [junit4]   2> 	at org.apache.solr.cloud.hdfs.HdfsTestUtil.teardownClass(HdfsTestUtil.java:197)
   [junit4]   2> 	at org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.teardownClass(HdfsCollectionsAPIDistributedZkTest.java:50)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   [junit4]   2> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2> 	at java.lang.reflect.Method.invoke(Method.java:483)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:799)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> Caused by: java.lang.NullPointerException
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.blockmanagement.BlocksMap.size(BlocksMap.java:198)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.getTotalBlocks(BlockManager.java:3291)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlocksTotal(FSNamesystem.java:6223)
   [junit4]   2> 	... 53 more
   [junit4]   2> 2991239 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[BFEA587A71B28EC1]-worker) [n:127.0.0.1:59625_ c:addReplicaColl s:shard1 r:core_node7 x:propertyDotName] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> Jul 11, 2015 2:22:33 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 2 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene53): {other_tl1=BlockTreeOrds(blocksize=128), range_facet_l_dv=PostingsFormat(name=Memory doPackFST= true), _version_=BlockTreeOrds(blocksize=128), multiDefault=PostingsFormat(name=Asserting), a_t=BlockTreeOrds(blocksize=128), intDefault=BlockTreeOrds(blocksize=128), a_i1=BlockTreeOrds(blocksize=128), id=PostingsFormat(name=Memory doPackFST= true), range_facet_i_dv=BlockTreeOrds(blocksize=128), text=PostingsFormat(name=LuceneVarGapDocFreqInterval), range_facet_l=BlockTreeOrds(blocksize=128), timestamp=BlockTreeOrds(blocksize=128)}, docValues:{range_facet_l_dv=DocValuesFormat(name=Direct), range_facet_i_dv=DocValuesFormat(name=Lucene50), timestamp=DocValuesFormat(name=Lucene50)}, sim=DefaultSimilarity, locale=zh_CN, timezone=Africa/Gaborone
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.8.0_25 (64-bit)/cpus=4,threads=2,free=164315896,total=531103744
   [junit4]   2> NOTE: All tests run in this JVM: [DirectUpdateHandlerTest, SimpleFacetsTest, PrimUtilsTest, ActionThrottleTest, TestBM25SimilarityFactory, TestExtendedDismaxParser, HardAutoCommitTest, CloudExitableDirectoryReaderTest, DirectUpdateHandlerOptimizeTest, SearchHandlerTest, TestPerFieldSimilarity, HdfsDirectoryTest, SynonymTokenizerTest, TestStressRecovery, TestCopyFieldCollectionResource, TestManagedResourceStorage, QueryParsingTest, TestSystemIdResolver, SolrInfoMBeanTest, StressHdfsTest, AtomicUpdatesTest, TestSolrConfigHandlerConcurrent, TestSolr4Spatial2, CollectionReloadTest, TestFieldTypeCollectionResource, VersionInfoTest, DistribJoinFromCollectionTest, TestMiniSolrCloudClusterSSL, TestHdfsUpdateLog, TestFieldCollectionResource, TestSearcherReuse, MoreLikeThisHandlerTest, SpellCheckCollatorTest, TestSolrJ, TestXIncludeConfig, NumericFieldsTest, TestHashQParserPlugin, RecoveryAfterSoftCommitTest, FullSolrCloudDistribCmdsTest, TestUniqueKeyFieldResource, TestObjectReleaseTracker, HdfsCollectionsAPIDistributedZkTest]
   [junit4] Completed [344/514] on J2 in 2221.17s, 1 test, 1 error <<< FAILURES!

[...truncated 561 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:533: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:474: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:61: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/extra-targets.xml:39: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build.xml:230: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/common-build.xml:513: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/common-build.xml:1426: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/common-build.xml:981: There were test failures: 514 suites, 2022 tests, 3 errors, 20 ignored (10 assumptions)

Total time: 174 minutes 37 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
No prior successful build to compare, so performing full copy of artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure