You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2016/06/24 08:02:35 UTC

[JENKINS] Lucene-Solr-NightlyTests-5.5 - Build # 19 - Still Failing

Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.5/19/

1 tests failed.
FAILED:  org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=130784, name=collection3, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=130784, name=collection3, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:58658/mn/l: collection already exists: awholynewstresscollection_collection3_1
	at __randomizedtesting.SeedInfo.seed([CB8363A083754520]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:577)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1100)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1575)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1596)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:984)




Build Log:
[...truncated 12550 lines...]
   [junit4] Suite: org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_CB8363A083754520-001/init-core-data-001
   [junit4]   2> 3795446 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[CB8363A083754520]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false)
   [junit4]   2> 3795446 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[CB8363A083754520]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /mn/l
   [junit4]   1> Formatting using clusterid: testClusterID
   [junit4]   2> 3795480 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[CB8363A083754520]-worker) [    ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
   [junit4]   2> 3795486 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[CB8363A083754520]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 3795488 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[CB8363A083754520]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 3795500 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[CB8363A083754520]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/hdfs to ./temp/Jetty_localhost_49962_hdfs____154wca/webapp
   [junit4]   2> 3795599 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[CB8363A083754520]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 3795806 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[CB8363A083754520]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:49962
   [junit4]   2> 3795865 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[CB8363A083754520]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 3795866 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[CB8363A083754520]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 3795874 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[CB8363A083754520]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_47269_datanode____.iwljpj/webapp
   [junit4]   2> 3795983 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[CB8363A083754520]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 3796275 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[CB8363A083754520]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:47269
   [junit4]   2> 3796295 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[CB8363A083754520]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 3796296 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[CB8363A083754520]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 3796308 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[CB8363A083754520]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_52362_datanode____xk5p3t/webapp
   [junit4]   2> 3796359 INFO  (IPC Server handler 3 on 56311) [    ] BlockStateChange BLOCK* processReport: from storage DS-b7012733-5ec2-47b7-a19e-2026b53c2f84 node DatanodeRegistration(127.0.0.1, datanodeUuid=5de52943-ebf7-44b2-9eaf-75d5100a6c45, infoPort=47269, ipcPort=34281, storageInfo=lv=-56;cid=testClusterID;nsid=673753889;c=0), blocks: 0, hasStaleStorages: true, processing time: 0 msecs
   [junit4]   2> 3796360 INFO  (IPC Server handler 3 on 56311) [    ] BlockStateChange BLOCK* processReport: from storage DS-4bc2004d-1631-4562-a983-44424a426be7 node DatanodeRegistration(127.0.0.1, datanodeUuid=5de52943-ebf7-44b2-9eaf-75d5100a6c45, infoPort=47269, ipcPort=34281, storageInfo=lv=-56;cid=testClusterID;nsid=673753889;c=0), blocks: 0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 3796413 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[CB8363A083754520]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 3796693 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[CB8363A083754520]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:52362
   [junit4]   2> 3796738 INFO  (IPC Server handler 8 on 56311) [    ] BlockStateChange BLOCK* processReport: from storage DS-b6404081-cac8-40ca-baea-f8e9edbbc86f node DatanodeRegistration(127.0.0.1, datanodeUuid=76b57766-c952-4ca7-b6bb-d927265ec4c0, infoPort=52362, ipcPort=32806, storageInfo=lv=-56;cid=testClusterID;nsid=673753889;c=0), blocks: 0, hasStaleStorages: true, processing time: 0 msecs
   [junit4]   2> 3796738 INFO  (IPC Server handler 8 on 56311) [    ] BlockStateChange BLOCK* processReport: from storage DS-ce96e45a-62be-496c-b195-13ddb28ce2b3 node DatanodeRegistration(127.0.0.1, datanodeUuid=76b57766-c952-4ca7-b6bb-d927265ec4c0, infoPort=52362, ipcPort=32806, storageInfo=lv=-56;cid=testClusterID;nsid=673753889;c=0), blocks: 0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 3796833 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 3796833 INFO  (Thread-18239) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 3796833 INFO  (Thread-18239) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 3796933 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.c.ZkTestServer start zk server on port:52129
   [junit4]   2> 3796933 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3796934 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3796935 INFO  (zkCallback-30454-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7c3d262b name:ZooKeeperConnection Watcher:127.0.0.1:52129 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3796935 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3796936 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3796936 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 3796937 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3796938 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3796938 INFO  (zkCallback-30455-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2b67047e name:ZooKeeperConnection Watcher:127.0.0.1:52129/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3796938 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3796938 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3796938 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 3796940 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 3796941 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 3796941 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 3796942 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 3796942 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 3796943 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 3796943 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 3796944 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 3796944 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 3796945 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 3796945 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 3796945 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 3796945 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 3796946 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 3796946 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 3796947 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 3796947 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 3796948 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 3796948 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 3796948 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 3796948 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 3796949 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 3796949 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 3796950 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 3796950 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 3796951 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3796951 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3796952 INFO  (zkCallback-30456-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6c7e2fc3 name:ZooKeeperConnection Watcher:127.0.0.1:52129 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3796952 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3796952 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3796952 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 3796953 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3796953 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3796953 INFO  (zkCallback-30457-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@265c3799 name:ZooKeeperConnection Watcher:127.0.0.1:52129/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3796953 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3796954 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3796954 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf2/solrconfig.xml
   [junit4]   2> 3796954 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/solrconfig.xml
   [junit4]   2> 3796955 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf2/schema.xml
   [junit4]   2> 3796955 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/schema.xml
   [junit4]   2> 3796956 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf2/enumsConfig.xml
   [junit4]   2> 3796956 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/enumsConfig.xml
   [junit4]   2> 3796957 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf2/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 3796957 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 3796958 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf2/stopwords.txt
   [junit4]   2> 3796958 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/stopwords.txt
   [junit4]   2> 3796959 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf2/protwords.txt
   [junit4]   2> 3796959 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/protwords.txt
   [junit4]   2> 3796959 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf2/currency.xml
   [junit4]   2> 3796960 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/currency.xml
   [junit4]   2> 3796960 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf2/open-exchange-rates.json
   [junit4]   2> 3796960 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/open-exchange-rates.json
   [junit4]   2> 3796961 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf2/mapping-ISOLatin1Accent.txt
   [junit4]   2> 3796961 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/mapping-ISOLatin1Accent.txt
   [junit4]   2> 3796962 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf2/old_synonyms.txt
   [junit4]   2> 3796962 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/old_synonyms.txt
   [junit4]   2> 3796963 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf2/synonyms.txt
   [junit4]   2> 3796963 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/synonyms.txt
   [junit4]   2> 3796964 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/core/src/test-files/solr/collection1/conf/elevate.xml to /configs/conf2/elevate.xml
   [junit4]   2> 3796964 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/elevate.xml
   [junit4]   2> 3797031 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_CB8363A083754520-001/control-001/cores/collection1
   [junit4]   2> 3797034 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 3797039 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5fd37370{/mn/l,null,AVAILABLE}
   [junit4]   2> 3797039 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.e.j.s.ServerConnector Started ServerConnector@2a71dc83{HTTP/1.1}{127.0.0.1:58658}
   [junit4]   2> 3797040 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.e.j.s.Server Started @3799795ms
   [junit4]   2> 3797040 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostPort=58658, solr.data.dir=hdfs://localhost:56311/hdfs__localhost_56311__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-5.5_solr_build_solr-core_test_J1_temp_solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_CB8363A083754520-001_tempDir-002_control_data, hostContext=/mn/l, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_CB8363A083754520-001/control-001/cores}
   [junit4]   2> 3797040 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@3827de24
   [junit4]   2> 3797040 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_CB8363A083754520-001/control-001'
   [junit4]   2> 3797041 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 3797041 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 3797041 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3797041 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3797042 INFO  (zkCallback-30458-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@74e46485 name:ZooKeeperConnection Watcher:127.0.0.1:52129/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3797042 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3797042 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3797043 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 3797043 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_CB8363A083754520-001/control-001/solr.xml
   [junit4]   2> 3797057 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_CB8363A083754520-001/control-001/cores
   [junit4]   2> 3797057 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.c.CoreContainer New CoreContainer 243778648
   [junit4]   2> 3797057 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_CB8363A083754520-001/control-001]
   [junit4]   2> 3797058 WARN  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_CB8363A083754520-001/control-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_CB8363A083754520-001/control-001/lib
   [junit4]   2> 3797058 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 3797062 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 3797065 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 3797066 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 3797066 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:52129/solr
   [junit4]   2> 3797067 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 3797067 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3797069 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3797072 INFO  (zkCallback-30461-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7d00b8e4 name:ZooKeeperConnection Watcher:127.0.0.1:52129 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3797073 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3797075 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3797078 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [n:127.0.0.1:58658_mn%2Fl    ] o.a.s.c.ZkController Added new OnReconnect listener org.apache.solr.cloud.ZkController$6@7b45d3c4
   [junit4]   2> 3797080 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [n:127.0.0.1:58658_mn%2Fl    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3797083 INFO  (zkCallback-30462-thread-1-processing-n:127.0.0.1:58658_mn%2Fl) [n:127.0.0.1:58658_mn%2Fl    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@d509a67 name:ZooKeeperConnection Watcher:127.0.0.1:52129/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3797083 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [n:127.0.0.1:58658_mn%2Fl    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3797086 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [n:127.0.0.1:58658_mn%2Fl    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 3797089 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [n:127.0.0.1:58658_mn%2Fl    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 3797092 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [n:127.0.0.1:58658_mn%2Fl    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 3797094 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [n:127.0.0.1:58658_mn%2Fl    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 3797097 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [n:127.0.0.1:58658_mn%2Fl    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 3797098 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [n:127.0.0.1:58658_mn%2Fl    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 3797100 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [n:127.0.0.1:58658_mn%2Fl    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 3797101 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [n:127.0.0.1:58658_mn%2Fl    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 3797102 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [n:127.0.0.1:58658_mn%2Fl    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 3797103 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [n:127.0.0.1:58658_mn%2Fl    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:58658_mn%2Fl
   [junit4]   2> 3797103 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [n:127.0.0.1:58658_mn%2Fl    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:58658_mn%2Fl
   [junit4]   2> 3797104 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [n:127.0.0.1:58658_mn%2Fl    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 3797106 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [n:127.0.0.1:58658_mn%2Fl    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 3797108 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [n:127.0.0.1:58658_mn%2Fl    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 3797108 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [n:127.0.0.1:58658_mn%2Fl    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/96125192600223750-127.0.0.1:58658_mn%2Fl-n_0000000000
   [junit4]   2> 3797108 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [n:127.0.0.1:58658_mn%2Fl    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:58658_mn%2Fl
   [junit4]   2> 3797108 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [n:127.0.0.1:58658_mn%2Fl    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 3797109 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [n:127.0.0.1:58658_mn%2Fl    ] o.a.s.c.Overseer Overseer (id=96125192600223750-127.0.0.1:58658_mn%2Fl-n_0000000000) starting
   [junit4]   2> 3797111 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [n:127.0.0.1:58658_mn%2Fl    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 3797114 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [n:127.0.0.1:58658_mn%2Fl    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 3797115 INFO  (OverseerStateUpdate-96125192600223750-127.0.0.1:58658_mn%2Fl-n_0000000000) [n:127.0.0.1:58658_mn%2Fl    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 3797115 INFO  (OverseerStateUpdate-96125192600223750-127.0.0.1:58658_mn%2Fl-n_0000000000) [n:127.0.0.1:58658_mn%2Fl    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 3797115 INFO  (OverseerCollectionConfigSetProcessor-96125192600223750-127.0.0.1:58658_mn%2Fl-n_0000000000) [n:127.0.0.1:58658_mn%2Fl    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 3797117 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [n:127.0.0.1:58658_mn%2Fl    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 3797143 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [n:127.0.0.1:58658_mn%2Fl    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 3797143 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [n:127.0.0.1:58658_mn%2Fl    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 3797144 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [n:127.0.0.1:58658_mn%2Fl    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_CB8363A083754520-001/control-001/cores
   [junit4]   2> 3797144 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [n:127.0.0.1:58658_mn%2Fl    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {config=solrconfig.xml, configSetProperties=configsetprops.json, shard=, collection=control_collection, schema=schema.xml, loadOnStartup=true, name=collection1, dataDir=data/, coreNodeName=, transient=false}
   [junit4]   2> 3797144 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [n:127.0.0.1:58658_mn%2Fl    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_CB8363A083754520-001/control-001/cores/collection1
   [junit4]   2> 3797144 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [n:127.0.0.1:58658_mn%2Fl    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 3797145 INFO  (coreLoadExecutor-13156-thread-1-processing-n:127.0.0.1:58658_mn%2Fl) [n:127.0.0.1:58658_mn%2Fl c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 3797145 INFO  (coreLoadExecutor-13156-thread-1-processing-n:127.0.0.1:58658_mn%2Fl) [n:127.0.0.1:58658_mn%2Fl c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 3797149 INFO  (coreLoadExecutor-13156-thread-1-processing-n:127.0.0.1:58658_mn%2Fl) [n:127.0.0.1:58658_mn%2Fl c:control_collection   x:collection1] o.a.s.c.ZkController Collection control_collection not visible yet, but flagging it so a watch is registered when it becomes visible
   [junit4]   2> 3797149 INFO  (coreLoadExecutor-13156-thread-1-processing-n:127.0.0.1:58658_mn%2Fl) [n:127.0.0.1:58658_mn%2Fl c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader addZkWatch [control_collection]
   [junit4]   2> 3797150 INFO  (coreLoadExecutor-13156-thread-1-processing-n:127.0.0.1:58658_mn%2Fl) [n:127.0.0.1:58658_mn%2Fl c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Deleting data for [control_collection]
   [junit4]   2> 3797150 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [n:127.0.0.1:58658_mn%2Fl    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/build/solr-core/test/J1
   [junit4]   2> 3797150 INFO  (OverseerStateUpdate-96125192600223750-127.0.0.1:58658_mn%2Fl-n_0000000000) [n:127.0.0.1:58658_mn%2Fl    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "state":"down",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:58658/mn/l",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "node_name":"127.0.0.1:58658_mn%2Fl",
   [junit4]   2>   "core":"collection1"} current state version: 0
   [junit4]   2> 3797150 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [n:127.0.0.1:58658_mn%2Fl    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 3797151 INFO  (OverseerStateUpdate-96125192600223750-127.0.0.1:58658_mn%2Fl-n_0000000000) [n:127.0.0.1:58658_mn%2Fl    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "state":"down",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:58658/mn/l",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "node_name":"127.0.0.1:58658_mn%2Fl",
   [junit4]   2>   "core":"collection1"}
   [junit4]   2> 3797156 INFO  (coreLoadExecutor-13156-thread-1-processing-n:127.0.0.1:58658_mn%2Fl) [n:127.0.0.1:58658_mn%2Fl c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 3797156 INFO  (OverseerStateUpdate-96125192600223750-127.0.0.1:58658_mn%2Fl-n_0000000000) [n:127.0.0.1:58658_mn%2Fl    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 3797156 INFO  (OverseerStateUpdate-96125192600223750-127.0.0.1:58658_mn%2Fl-n_0000000000) [n:127.0.0.1:58658_mn%2Fl    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 3797158 INFO  (zkCallback-30462-thread-1-processing-n:127.0.0.1:58658_mn%2Fl) [n:127.0.0.1:58658_mn%2Fl    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 3797158 INFO  (zkCallback-30462-thread-2-processing-n:127.0.0.1:58658_mn%2Fl) [n:127.0.0.1:58658_mn%2Fl    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 3798156 INFO  (coreLoadExecutor-13156-thread-1-processing-n:127.0.0.1:58658_mn%2Fl) [n:127.0.0.1:58658_mn%2Fl c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 3798157 INFO  (coreLoadExecutor-13156-thread-1-processing-n:127.0.0.1:58658_mn%2Fl) [n:127.0.0.1:58658_mn%2Fl c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 3798157 INFO  (coreLoadExecutor-13156-thread-1-processing-n:127.0.0.1:58658_mn%2Fl) [n:127.0.0.1:58658_mn%2Fl c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 3798157 INFO  (coreLoadExecutor-13156-thread-1-processing-n:127.0.0.1:58658_mn%2Fl) [n:127.0.0.1:58658_mn%2Fl c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/control_collection]
   [junit4]   2> 3798158 INFO  (coreLoadExecutor-13156-thread-1-processing-n:127.0.0.1:58658_mn%2Fl) [n:127.0.0.1:58658_mn%2Fl c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/control_collection] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 3798158 INFO  (coreLoadExecutor-13156-thread-1-processing-n:127.0.0.1:58658_mn%2Fl) [n:127.0.0.1:58658_mn%2Fl c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_CB8363A083754520-001/control-001/cores/collection1'
   [junit4]   2> 3798158 INFO  (coreLoadExecutor-13156-thread-1-processing-n:127.0.0.1:58658_mn%2Fl) [n:127.0.0.1:58658_mn%2Fl c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 3798158 INFO  (coreLoadExecutor-13156-thread-1-processing-n:127.0.0.1:58658_mn%2Fl) [n:127.0.0.1:58658_mn%2Fl c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 3798160 INFO  (coreLoadExecutor-13156-thread-1-processing-n:127.0.0.1:58658_mn%2Fl) [n:127.0.0.1:58658_mn%2Fl c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 3798164 INFO  (coreLoadExecutor-13156-thread-1-processing-n:127.0.0.1:58658_mn%2Fl) [n:127.0.0.1:58658_mn%2Fl c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 3798173 INFO  (coreLoadExecutor-13156-thread-1-processing-n:127.0.0.1:58658_mn%2Fl) [n:127.0.0.1:58658_mn%2Fl c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.5.2
   [junit4]   2> 3798192 INFO  (coreLoadExecutor-13156-thread-1-processing-n:127.0.0.1:58658_mn%2Fl) [n:127.0.0.1:58658_mn%2Fl c:control_collection   x:collection1] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 3798197 INFO  (coreLoadExecutor-13156-thread-1-processing-n:127.0.0.1:58658_mn%2Fl) [n:127.0.0.1:58658_mn%2Fl c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 3799182 ERROR (Finalizer) [    ] o.a.s.c.CoreContainer CoreContainer was not close prior to finalize(), indicates a bug -- POSSIBLE RESOURCE LEAK!!!  instance=1738048980
   [junit4]   2> 3799354 INFO  (coreLoadExecutor-13156-thread-1-processing-n:127.0.0.1:58658_mn%2Fl) [n:127.0.0.1:58658_mn%2Fl c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 3799364 INFO  (coreLoadExecutor-13156-thread-1-processing-n:127.0.0.1:58658_mn%2Fl) [n:127.0.0.1:58658_mn%2Fl c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 3799366 INFO  (coreLoadExecutor-13156-thread-1-processing-n:127.0.0.1:58658_mn%2Fl) [n:127.0.0.1:58658_mn%2Fl c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 3799375 INFO  (coreLoadExecutor-13156-thread-1-processing-n:127.0.0.1:58658_mn%2Fl) [n:127.0.0.1:58658_mn%2Fl c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 3799379 INFO  (coreLoadExecutor-13156-thread-1-processing-n:127.0.0.1:58658_mn%2Fl) [n:127.0.0.1:58658_mn%2Fl c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 3799381 INFO  (coreLoadExecutor-13156-thread-1-processing-n:127.0.0.1:58658_mn%2Fl) [n:127.0.0.1:58658_mn%2Fl c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 3799381 INFO  (coreLoadExecutor-13156-thread-1-processing-n:127.0.0.1:58658_mn%2Fl) [n:127.0.0.1:58658_mn%2Fl c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 3799383 INFO  (coreLoadExecutor-13156-thread-1-processing-n:127.0.0.1:58658_mn%2Fl) [n:127.0.0.1:58658_mn%2Fl c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/build/solr-core/test/J1
   [junit4]   2> 3799383 INFO  (coreLoadExecutor-13156-thread-1-processing-n:127.0.0.1:58658_mn%2Fl) [n:127.0.0.1:58658_mn%2Fl c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 3799383 INFO  (coreLoadExecutor-13156-thread-1-processing-n:127.0.0.1:58658_mn%2Fl) [n:127.0.0.1:58658_mn%2Fl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.HdfsDirectoryFactory
   [junit4]   2> 3799383 INFO  (coreLoadExecutor-13156-thread-1-processing-n:127.0.0.1:58658_mn%2Fl) [n:127.0.0.1:58658_mn%2Fl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:56311/solr_hdfs_home
   [junit4]   2> 3799383 INFO  (coreLoadExecutor-13156-thread-1-processing-n:127.0.0.1:58658_mn%2Fl) [n:127.0.0.1:58658_mn%2Fl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
   [junit4]   2> 3799383 INFO  (coreLoadExecutor-13156-thread-1-processing-n:127.0.0.1:58658_mn%2Fl) [n:127.0.0.1:58658_mn%2Fl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_CB8363A083754520-001/control-001/cores/collection1], dataDir=[hdfs://localhost:56311/solr_hdfs_home/control_collection/core_node1/data/]
   [junit4]   2> 3799383 INFO  (coreLoadExecutor-13156-thread-1-processing-n:127.0.0.1:58658_mn%2Fl) [n:127.0.0.1:58658_mn%2Fl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@60ab1380
   [junit4]   2> 3799384 INFO  (coreLoadExecutor-13156-thread-1-processing-n:127.0.0.1:58658_mn%2Fl) [n:127.0.0.1:58658_mn%2Fl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:56311/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 3799396 INFO  (coreLoadExecutor-13156-thread-1-processing-n:127.0.0.1:58658_mn%2Fl) [n:127.0.0.1:58658_mn%2Fl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:56311/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 3799397 INFO  (coreLoadExecutor-13156-thread-1-processing-n:127.0.0.1:58658_mn%2Fl) [n:127.0.0.1:58658_mn%2Fl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=hdfs://localhost:56311/solr_hdfs_home/control_collection/core_node1/data/index/
   [junit4]   2> 3799405 WARN  (coreLoadExecutor-13156-thread-1-processing-n:127.0.0.1:58658_mn%2Fl) [n:127.0.0.1:58658_mn%2Fl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory 'hdfs:/localhost:56311/solr_hdfs_home/control_collection/core_node1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 3799405 INFO  (coreLoadExecutor-13156-thread-1-processing-n:127.0.0.1:58658_mn%2Fl) [n:127.0.0.1:58658_mn%2Fl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:56311/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 3799415 INFO  (coreLoadExecutor-13156-thread-1-processing-n:127.0.0.1:58658_mn%2Fl) [n:127.0.0.1:58658_mn%2Fl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:56311/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 3799416 INFO  (coreLoadExecutor-13156-thread-1-processing-n:127.0.0.1:58658_mn%2Fl) [n:127.0.0.1:58658_mn%2Fl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=9, maxMergeAtOnceExplicit=8, maxMergedSegmentMB=1.2841796875, floorSegmentMB=0.25390625, forceMergeDeletesPctAllowed=5.759116282312718, segmentsPerTier=11.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
   [junit4]   2> 3799462 INFO  (IPC Server handler 5 on 56311) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:57089 is added to blk_1073741825_1001{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-b7012733-5ec2-47b7-a19e-2026b53c2f84:NORMAL:127.0.0.1:55499|RBW], ReplicaUnderConstruction[[DISK]DS-ce96e45a-62be-496c-b195-13ddb28ce2b3:NORMAL:127.0.0.1:57089|FINALIZED]]} size 0
   [junit4]   2> 3799463 INFO  (IPC Server handler 4 on 56311) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:55499 is added to blk_1073741825_1001{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-b7012733-5ec2-47b7-a19e-2026b53c2f84:NORMAL:127.0.0.1:55499|RBW], ReplicaUnderConstruction[[DISK]DS-ce96e45a-62be-496c-b195-13ddb28ce2b3:NORMAL:127.0.0.1:57089|FINALIZED]]} size 0
   [junit4]   2> 3799465 INFO  (coreLoadExecutor-13156-thread-1-processing-n:127.0.0.1:58658_mn%2Fl) [n:127.0.0.1:58658_mn%2Fl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(HdfsDirectory@hdfs://localhost:56311/solr_hdfs_home/control_collection/core_node1/data/index lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1fe35de6; maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 3799465 INFO  (coreLoadExecutor-13156-thread-1-processing-n:127.0.0.1:58658_mn%2Fl) [n:127.0.0.1:58658_mn%2Fl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 3799466 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:58658_mn%2Fl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in hdfs://localhost:56311/solr_hdfs_home/control_collection/core_node1/data/
   [junit4]   2> 3799468 INFO  (coreLoadExecutor-13156-thread-1-processing-n:127.0.0.1:58658_mn%2Fl) [n:127.0.0.1:58658_mn%2Fl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 3799468 INFO  (coreLoadExecutor-13156-thread-1-processing-n:127.0.0.1:58658_mn%2Fl) [n:127.0.0.1:58658_mn%2Fl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 3799468 INFO  (coreLoadExecutor-13156-thread-1-processing-n:127.0.0.1:58658_mn%2Fl) [n:127.0.0.1:58658_mn%2Fl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 3799468 INFO  (coreLoadExecutor-13156-thread-1-processing-n:127.0.0.1:58658_mn%2Fl) [n:127.0.0.1:58658_mn%2Fl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 3799468 INFO  (coreLoadExecutor-13156-thread-1-processing-n:127.0.0.1:58658_mn%2Fl) [n:127.0.0.1:58658_mn%2Fl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 3799468 INFO  (coreLoadExecutor-13156-thread-1-processing-n:127.0.0.1:58658_mn%2Fl) [n:127.0.0.1:58658_mn%2Fl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 3799468 INFO  (coreLoadExecutor-13156-thread-1-processing-n:127.0.0.1:58658_mn%2Fl) [n:127.0.0.1:58658_mn%2Fl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 3799468 INFO  (coreLoadExecutor-13156-thread-1-processing-n:127.0.0.1:58658_mn%2Fl) [n:127.0.0.1:58658_mn%2Fl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 3799468 INFO  (coreLoadExecutor-13156-thread-1-processing-n:127.0.0.1:58658_mn%2Fl) [n:127.0.0.1:58658_mn%2Fl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 3799468 WARN  (coreLoadExecutor-13156-thread-1-processing-n:127.0.0.1:58658_mn%2Fl) [n:127.0.0.1:58658_mn%2Fl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
   [junit4]   2> 3799470 INFO  (coreLoadExecutor-13156-thread-1-processing-n:127.0.0.1:58658_mn%2Fl) [n:127.0.0.1:58658_mn%2Fl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 3799471 INFO  (coreLoadExecutor-13156-thread-1-processing-n:127.0.0.1:58658_mn%2Fl) [n:127.0.0.1:58658_mn%2Fl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 3799472 INFO  (coreLoadExecutor-13156-thread-1-processing-n:127.0.0.1:58658_mn%2Fl) [n:127.0.0.1:58658_mn%2Fl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 3799473 INFO  (coreLoadExecutor-13156-thread-1-processing-n:127.0.0.1:58658_mn%2Fl) [n:127.0.0.1:58658_mn%2Fl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 3799477 INFO  (coreLoadExecutor-13156-thread-1-processing-n:127.0.0.1:58658_mn%2Fl) [n:127.0.0.1:58658_mn%2Fl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/threads,/update/json/docs,/admin/luke,/admin/file,standard,/admin/mbeans,/replication,/dump,/admin/logging,/update,/get,/update/json,/admin/plugins,/schema,/admin/properties,/admin/system,/admin/segments,/config,/update/csv,/admin/ping
   [junit4]   2> 3799478 INFO  (coreLoadExecutor-13156-thread-1-processing-n:127.0.0.1:58658_mn%2Fl) [n:127.0.0.1:58658_mn%2Fl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 3799478 INFO  (coreLoadExecutor-13156-thread-1-processing-n:127.0.0.1:58658_mn%2Fl) [n:127.0.0.1:58658_mn%2Fl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
   [junit4]   2> 3799478 INFO  (coreLoadExecutor-13156-thread-1-processing-n:127.0.0.1:58658_mn%2Fl) [n:127.0.0.1:58658_mn%2Fl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 3799478 INFO  (coreLoadExecutor-13156-thread-1-processing-n:127.0.0.1:58658_mn%2Fl) [n:127.0.0.1:58658_mn%2Fl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=2
   [junit4]   2> 3799487 INFO  (coreLoadExecutor-13156-thread-1-processing-n:127.0.0.1:58658_mn%2Fl) [n:127.0.0.1:58658_mn%2Fl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 3799487 INFO  (coreLoadExecutor-13156-thread-1-processing-n:127.0.0.1:58658_mn%2Fl) [n:127.0.0.1:58658_mn%2Fl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 3799491 INFO  (coreLoadExecutor-13156-thread-1-processing-n:127.0.0.1:58658_mn%2Fl) [n:127.0.0.1:58658_mn%2Fl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=22, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.799049105982103]
   [junit4]   2> 3799500 INFO  (coreLoadExecutor-13156-thread-1-processing-n:127.0.0.1:58658_mn%2Fl) [n:127.0.0.1:58658_mn%2Fl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(HdfsDirectory@hdfs://localhost:56311/solr_hdfs_home/control_collection/core_node1/data/index lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1fe35de6; maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 3799500 INFO  (coreLoadExecutor-13156-thread-1-processing-n:127.0.0.1:58658_mn%2Fl) [n:127.0.0.1:58658_mn%2Fl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 3799500 INFO  (coreLoadExecutor-13156-thread-1-processing-n:127.0.0.1:58658_mn%2Fl) [n:127.0.0.1:58658_mn%2Fl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@aad80ce[collection1] main
   [junit4]   2> 3799500 INFO  (coreLoadExecutor-13156-thread-1-processing-n:127.0.0.1:58658_mn%2Fl) [n:127.0.0.1:58658_mn%2Fl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/control_collection]
   [junit4]   2> 3799501 INFO  (coreLoadExecutor-13156-thread-1-processing-n:127.0.0.1:58658_mn%2Fl) [n:127.0.0.1:58658_mn%2Fl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/control_collection] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 3799501 INFO  (coreLoadExecutor-13156-thread-1-processing-n:127.0.0.1:58658_mn%2Fl) [n:127.0.0.1:58658_mn%2Fl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 3799501 INFO  (coreLoadExecutor-13156-thread-1-processing-n:127.0.0.1:58658_mn%2Fl) [n:127.0.0.1:58658_mn%2Fl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 3799501 INFO  (coreLoadExecutor-13156-thread-1-processing-n:127.0.0.1:58658_mn%2Fl) [n:127.0.0.1:58658_mn%2Fl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 3799501 INFO  (coreLoadExecutor-13156-thread-1-processing-n:127.0.0.1:58658_mn%2Fl) [n:127.0.0.1:58658_mn%2Fl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 3799501 INFO  (coreLoadExecutor-13156-thread-1-processing-n:127.0.0.1:58658_mn%2Fl) [n:127.0.0.1:58658_mn%2Fl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 3799501 INFO  (coreLoadExecutor-13156-thread-1-processing-n:127.0.0.1:58658_mn%2Fl) [n:127.0.0.1:58658_mn%2Fl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 3799502 INFO  (coreLoadExecutor-13156-thread-1-processing-n:127.0.0.1:58658_mn%2Fl) [n:127.0.0.1:58658_mn%2Fl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 3799502 INFO  (coreLoadExecutor-13156-thread-1-processing-n:127.0.0.1:58658_mn%2Fl) [n:127.0.0.1:58658_mn%2Fl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 3799503 INFO  (searcherExecutor-13157-thread-1-processing-s:shard1 c:control_collection n:127.0.0.1:58658_mn%2Fl r:core_node1 x:collection1) [n:127.0.0.1:58658_mn%2Fl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@aad80ce[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 3799503 INFO  (coreLoadExecutor-13156-thread-1-processing-n:127.0.0.1:58658_mn%2Fl) [n:127.0.0.1:58658_mn%2Fl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 3799503 INFO  (coreLoadExecutor-13156-thread-1-processing-n:127.0.0.1:58658_mn%2Fl) [n:127.0.0.1:58658_mn%2Fl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 3799503 INFO  (coreLoadExecutor-13156-thread-1-processing-n:127.0.0.1:58658_mn%2Fl) [n:127.0.0.1:58658_mn%2Fl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 3799503 INFO  (coreLoadExecutor-13156-thread-1-processing-n:127.0.0.1:58658_mn%2Fl) [n:127.0.0.1:58658_mn%2Fl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1538003084402229248
   [junit4]   2> 3799505 INFO  (coreLoadExecutor-13156-thread-1-processing-n:127.0.0.1:58658_mn%2Fl) [n:127.0.0.1:58658_mn%2Fl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 1.0ms to seed version buckets with highest version 1538003084402229248
   [junit4]   2> 3799505 INFO  (coreLoadExecutor-13156-thread-1-processing-n:127.0.0.1:58658_mn%2Fl) [n:127.0.0.1:58658_mn%2Fl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 3799505 INFO  (coreLoadExecutor-13156-thread-1-processing-n:127.0.0.1:58658_mn%2Fl) [n:127.0.0.1:58658_mn%2Fl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 3799505 INFO  (coreZkRegister-13150-thread-1-processing-s:shard1 c:control_collection n:127.0.0.1:58658_mn%2Fl r:core_node1 x:collection1) [n:127.0.0.1:58658_mn%2Fl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:58658/mn/l collection:control_collection shard:shard1
   [junit4]   2> 3799506 INFO  (coreZkRegister-13150-thread-1-processing-s:shard1 c:control_collection n:127.0.0.1:58658_mn%2Fl r:core_node1 x:collection1) [n:127.0.0.1:58658_mn%2Fl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 3799507 INFO  (coreZkRegister-13150-thread-1-processing-s:shard1 c:control_collection n:127.0.0.1:58658_mn%2Fl r:core_node1 x:collection1) [n:127.0.0.1:58658_mn%2Fl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/96125192600223750-core_node1-n_0000000000
   [junit4]   2> 3799507 INFO  (coreZkRegister-13150-thread-1-processing-s:shard1 c:control_collection n:127.0.0.1:58658_mn%2Fl r:core_node1 x:collection1) [n:127.0.0.1:58658_mn%2Fl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard=shard1 and weAreReplacement=false and leaderVoteWait=180000
   [junit4]   2> 3799510 INFO  (coreZkRegister-13150-thread-1-processing-s:shard1 c:control_collection n:127.0.0.1:58658_mn%2Fl r:core_node1 x:collection1) [n:127.0.0.1:58658_mn%2Fl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 3799510 INFO  (OverseerStateUpdate-96125192600223750-127.0.0.1:58658_mn%2Fl-n_0000000000) [n:127.0.0.1:58658_mn%2Fl    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 3799510 INFO  (coreZkRegister-13150-thread-1-processing-s:shard1 c:control_collection n:127.0.0.1:58658_mn%2Fl r:core_node1 x:collection1) [n:127.0.0.1:58658_mn%2Fl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 3799510 INFO  (coreZkRegister-13150-thread-1-processing-s:shard1 c:control_collection n:127.0.0.1:58658_mn%2Fl r:core_node1 x:collection1) [n:127.0.0.1:58658_mn%2Fl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:58658/mn/l/collection1/
   [junit4]   2> 3799510 INFO  (coreZkRegister-13150-thread-1-processing-s:shard1 c:control_collection n:127.0.0.1:58658_mn%2Fl r:core_node1 x:collection1) [n:127.0.0.1:58658_mn%2Fl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 3799510 INFO  (zkCallback-30462-thread-1-processing-n:127.0.0.1:58658_mn%2Fl) [n:127.0.0.1:58658_mn%2Fl    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 3799510 INFO  (coreZkRegister-13150-thread-1-processing-s:shard1 c:control_collection n:127.0.0.1:58658_mn%2Fl r:core_node1 x:collection1) [n:127.0.0.1:58658_mn%2Fl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:58658/mn/l/collection1/ has no replicas
   [junit4]   2> 3799510 INFO  (zkCallback-30462-thread-3-processing-n:127.0.0.1:58658_mn%2Fl) [n:127.0.0.1:58658_mn%2Fl    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 3799512 INFO  (coreZkRegister-13150-thread-1-processing-s:shard1 c:control_collection n:127.0.0.1:58658_mn%2Fl r:core_node1 x:collection1) [n:127.0.0.1:58658_mn%2Fl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 3799513 INFO  (coreZkRegister-13150-thread-1-processing-s:shard1 c:control_collection n:127.0.0.1:58658_mn%2Fl r:core_node1 x:collection1) [n:127.0.0.1:58658_mn%2Fl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/control_collection/leaders/shard1/leader after winning as /collections/control_collection/leader_elect/shard1/election/96125192600223750-core_node1-n_0000000000
   [junit4]   2> 3799514 INFO  (coreZkRegister-13150-thread-1-processing-s:shard1 c:control_collection n:127.0.0.1:58658_mn%2Fl r:core_node1 x:collection1) [n:127.0.0.1:58658_mn%2Fl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:58658/mn/l/collection1/ shard1
   [junit4]   2> 3799515 INFO  (OverseerStateUpdate-96125192600223750-127.0.0.1:58658_mn%2Fl-n_0000000000) [n:127.0.0.1:58658_mn%2Fl    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:58658/mn/l",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 2
   [junit4]   2> 3799616 INFO  (zkCallback-30462-thread-1-processing-n:127.0.0.1:58658_mn%2Fl) [n:127.0.0.1:58658_mn%2Fl    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 3799617 INFO  (zkCallback-30462-thread-3-processing-n:127.0.0.1:58658_mn%2Fl) [n:127.0.0.1:58658_mn%2Fl    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 3799665 INFO  (coreZkRegister-13150-thread-1-processing-s:shard1 c:control_collection n:127.0.0.1:58658_mn%2Fl r:core_node1 x:collection1) [n:127.0.0.1:58658_mn%2Fl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:58658/mn/l/collection1/ and leader is http://127.0.0.1:58658/mn/l/collection1/
   [junit4]   2> 3799665 INFO  (coreZkRegister-13150-thread-1-processing-s:shard1 c:control_collection n:127.0.0.1:58658_mn%2Fl r:core_node1 x:collection1) [n:127.0.0.1:58658_mn%2Fl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:58658/mn/l
   [junit4]   2> 3799665 INFO  (coreZkRegister-13150-thread-1-processing-s:shard1 c:control_collection n:127.0.0.1:58658_mn%2Fl r:core_node1 x:collection1) [n:127.0.0.1:58658_mn%2Fl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 3799665 INFO  (coreZkRegister-13150-thread-1-processing-s:shard1 c:control_collection n:127.0.0.1:58658_mn%2Fl r:core_node1 x:collection1) [n:127.0.0.1:58658_mn%2Fl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 3799665 INFO  (coreZkRegister-13150-thread-1-processing-s:shard1 c:control_collection n:127.0.0.1:58658_mn%2Fl r:core_node1 x:collection1) [n:127.0.0.1:58658_mn%2Fl c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 3799667 INFO  (OverseerStateUpdate-96125192600223750-127.0.0.1:58658_mn%2Fl-n_0000000000) [n:127.0.0.1:58658_mn%2Fl    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "state":"active",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:58658/mn/l",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "node_name":"127.0.0.1:58658_mn%2Fl",
   [junit4]   2>   "core":"collection1"} current state version: 3
   [junit4]   2> 3799667 INFO  (OverseerStateUpdate-96125192600223750-127.0.0.1:58658_mn%2Fl-n_0000000000) [n:127.0.0.1:58658_mn%2Fl    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "state":"active",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:58658/mn/l",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "node_name":"127.0.0.1:58658_mn%2Fl",
   [junit4]   2>   "core":"collection1"}
   [junit4]   2> 3799675 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3799675 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3799676 INFO  (zkCallback-30464-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7ef1a486 name:ZooKeeperConnection Watcher:127.0.0.1:52129/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3799676 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3799676 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3799676 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 3799677 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 3799732 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_CB8363A083754520-001/shard-1-001/cores/collection1
   [junit4]   2> 3799733 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_CB8363A083754520-001/shard-1-001
   [junit4]   2> 3799733 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 3799735 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@7e04b841{/mn/l,null,AVAILABLE}
   [junit4]   2> 3799735 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.e.j.s.ServerConnector Started ServerConnector@5b19b024{HTTP/1.1}{127.0.0.1:59406}
   [junit4]   2> 3799735 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.e.j.s.Server Started @3802490ms
   [junit4]   2> 3799735 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/mn/l, hostPort=59406, solrconfig=solrconfig.xml, solr.data.dir=hdfs://localhost:56311/hdfs__localhost_56311__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-5.5_solr_build_solr-core_test_J1_temp_solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_CB8363A083754520-001_tempDir-002_jetty1, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_CB8363A083754520-001/shard-1-001/cores}
   [junit4]   2> 3799735 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@3827de24
   [junit4]   2> 3799735 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_CB8363A083754520-001/shard-1-001'
   [junit4]   2> 3799736 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 3799736 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 3799736 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3799736 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3799737 INFO  (zkCallback-30465-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@48b29374 name:ZooKeeperConnection Watcher:127.0.0.1:52129/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3799737 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3799737 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3799740 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 3799740 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_CB8363A083754520-001/shard-1-001/solr.xml
   [junit4]   2> 3799749 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_CB8363A083754520-001/shard-1-001/cores
   [junit4]   2> 3799749 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.c.CoreContainer New CoreContainer 496687040
   [junit4]   2> 3799749 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_CB8363A083754520-001/shard-1-001]
   [junit4]   2> 3799749 WARN  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_CB8363A083754520-001/shard-1-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_CB8363A083754520-001/shard-1-001/lib
   [junit4]   2> 3799749 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 3799750 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 3799751 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 3799751 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 3799751 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:52129/solr
   [junit4]   2> 3799751 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 3799751 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3799751 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3799752 INFO  (zkCallback-30468-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@eba99f5 name:ZooKeeperConnection Watcher:127.0.0.1:52129 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3799752 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3799752 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3799753 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [n:127.0.0.1:59406_mn%2Fl    ] o.a.s.c.ZkController Added new OnReconnect listener org.apache.solr.cloud.ZkController$6@54e132d0
   [junit4]   2> 3799753 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [n:127.0.0.1:59406_mn%2Fl    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3799754 INFO  (zkCallback-30469-thread-1-processing-n:127.0.0.1:59406_mn%2Fl) [n:127.0.0.1:59406_mn%2Fl    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6a4edcd8 name:ZooKeeperConnection Watcher:127.0.0.1:52129/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3799754 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [n:127.0.0.1:59406_mn%2Fl    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3799758 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[CB8363A083754520]) [n:127.0.0.1:59406_mn%2Fl    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 3799769 INFO  (zkCallback-30469-thread-1-processing-n:127.0.0.1:59406_mn%2Fl) [n:127.0.0.1:59406_mn%2Fl    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 3799769 INFO  (zkCallback-30462-thread-3-processing-n:127.0.0.1:58658_mn%2Fl) [n:127.0.0.1:58658_mn%2Fl    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 3799769 INFO  (zkCallback-30462-thread-2-processing-n:127.0.0.1:58658_mn%2Fl) [n:127.0.0.1:58658_mn%2Fl    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 3799769 INFO  (zkCallback-30464-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 3800761 INFO  (TEST-HdfsCo

[...truncated too long message...]

SystemPropertiesRestoreRule.java:57)
   [junit4]   2> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> Caused by: java.lang.NullPointerException
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.blockmanagement.BlocksMap.size(BlocksMap.java:198)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.getTotalBlocks(BlockManager.java:3291)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlocksTotal(FSNamesystem.java:6223)
   [junit4]   2> 	... 54 more
   [junit4]   2> 4185047 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[CB8363A083754520]-worker) [    ] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> Jun 24, 2016 7:47:05 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 3 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene54): {a_i1=BlockTreeOrds(blocksize=128), timestamp=BlockTreeOrds(blocksize=128), range_facet_l_dv=PostingsFormat(name=LuceneFixedGap), range_facet_l=BlockTreeOrds(blocksize=128), other_tl1=FSTOrd50, intDefault=FSTOrd50, id=PostingsFormat(name=LuceneFixedGap), multiDefault=BlockTreeOrds(blocksize=128), a_t=BlockTreeOrds(blocksize=128), range_facet_i_dv=BlockTreeOrds(blocksize=128), text=PostingsFormat(name=Direct), _version_=FSTOrd50}, docValues:{timestamp=DocValuesFormat(name=Direct), range_facet_l_dv=DocValuesFormat(name=Asserting), range_facet_i_dv=DocValuesFormat(name=Direct)}, sim=RandomSimilarity(queryNorm=false,coord=yes): {}, locale=it-IT, timezone=America/Argentina/Catamarca
   [junit4]   2> NOTE: Linux 3.13.0-85-generic amd64/Oracle Corporation 1.7.0_80 (64-bit)/cpus=4,threads=2,free=240671688,total=500170752
   [junit4]   2> NOTE: All tests run in this JVM: [ParsingFieldUpdateProcessorsTest, TestReplicationHandlerBackup, TestSolrConfigHandlerCloud, TestSimpleTrackingShardHandler, TestSystemIdResolver, DistributedIntervalFacetingTest, TestDistributedGrouping, SpellCheckComponentTest, TestCursorMarkWithoutUniqueKey, VersionInfoTest, HdfsUnloadDistributedZkTest, CreateCollectionCleanupTest, TestDocBasedVersionConstraints, TestSolrDeletionPolicy2, TestDocumentBuilder, TestRTimerTree, TestQueryTypes, NumericFieldsTest, ReplicationFactorTest, TestBinaryResponseWriter, BasicAuthIntegrationTest, ConvertedLegacyTest, TestLFUCache, SuggestComponentTest, MultiTermTest, TestRequestForwarding, TestRecoveryHdfs, TestClassNameShortening, TestDFRSimilarityFactory, BigEndianAscendingWordDeserializerTest, OverseerTaskQueueTest, LoggingHandlerTest, TestSolrCloudWithKerberosAlt, SharedFSAutoReplicaFailoverTest, TermVectorComponentTest, MBeansHandlerTest, CloudMLTQParserTest, TestCodecSupport, TestComplexPhraseQParserPlugin, MultiThreadedOCPTest, TestStressVersions, SolrCmdDistributorTest, SecurityConfHandlerTest, ChangedSchemaMergeTest, SolrIndexSplitterTest, TestTrie, CollectionTooManyReplicasTest, ChaosMonkeyNothingIsSafeTest, TestRemoveLastDynamicCopyField, TestLMJelinekMercerSimilarityFactory, UUIDUpdateProcessorFallbackTest, TestDynamicFieldCollectionResource, TestSolrQueryParserDefaultOperatorResource, TestSurroundQueryParser, AddBlockUpdateTest, IndexBasedSpellCheckerTest, OpenCloseCoreStressTest, MinimalSchemaTest, ZkControllerTest, FileUtilsTest, HdfsNNFailoverTest, RequestLoggingTest, OpenExchangeRatesOrgProviderTest, UniqFieldsUpdateProcessorFactoryTest, CloneFieldUpdateProcessorFactoryTest, HdfsRestartWhileUpdatingTest, CloudExitableDirectoryReaderTest, BufferStoreTest, DeleteStatusTest, AnalyticsMergeStrategyTest, DeleteLastCustomShardedReplicaTest, ForceLeaderTest, TestSolrQueryResponse, SolrIndexConfigTest, TimeZoneUtilsTest, SimplePostToolTest, TestCoreContainer, LeaderInitiatedRecoveryOnCommitTest, EnumFieldTest, SchemaVersionSpecificBehaviorTest, FacetPivotSmallTest, TestFastLRUCache, DistributedTermsComponentTest, ZkSolrClientTest, HighlighterMaxOffsetTest, SpellCheckCollatorTest, TestImplicitCoreProperties, BasicFunctionalityTest, TestStandardQParsers, CSVRequestHandlerTest, MergeStrategyTest, TestFiltering, OverseerTest, ClusterStateUpdateTest, TestZkChroot, TestRandomDVFaceting, TestRangeQuery, SimpleFacetsTest, TestSolr4Spatial, SolrCoreTest, TestSort, TestIndexSearcher, SoftAutoCommitTest, SuggesterTSTTest, SuggesterTest, SpatialFilterTest, SuggesterWFSTTest, TestCSVLoader, PolyFieldTest, WordBreakSolrSpellCheckerTest, TestPseudoReturnFields, TestUpdate, FieldMutatingUpdateProcessorTest, TestAtomicUpdateErrorCases, TestWordDelimiterFilterFactory, QueryEqualityTest, DefaultValueUpdateProcessorTest, StatelessScriptUpdateProcessorFactoryTest, SortByFunctionTest, TestRemoteStreaming, DocValuesMultiTest, DistanceFunctionTest, SolrInfoMBeanTest, XsltUpdateRequestHandlerTest, DebugComponentTest, LukeRequestHandlerTest, TestQueryUtils, DirectSolrSpellCheckerTest, DocumentAnalysisRequestHandlerTest, FileBasedSpellCheckerTest, TermsComponentTest, DocumentBuilderTest, TestValueSourceCache, PathHierarchyTokenizerFactoryTest, TestIndexingPerformance, MoreLikeThisHandlerTest, RequiredFieldsTest, FastVectorHighlighterTest, TestJmxIntegration, UpdateRequestProcessorFactoryTest, TestPartialUpdateDeduplication, HighlighterConfigTest, TestQuerySenderNoQuery, ResponseLogComponentTest, TestMergePolicyConfig, TestDocSet, TestSearchPerf, OutputWriterTest, TestFuzzyAnalyzedSuggestions, TestSolrCoreProperties, TestPostingsSolrHighlighter, DirectSolrConnectionTest, NotRequiredUniqueKeyTest, TestLuceneMatchVersion, SpellPossibilityIteratorTest, TestXIncludeConfig, TestLMDirichletSimilarityFactory, TestIBSimilarityFactory, TestBM25SimilarityFactory, ResourceLoaderTest, ScriptEngineTest, URLClassifyProcessorTest, PreAnalyzedFieldTest, TestSuggestSpellingConverter, SpellingQueryConverterTest, DOMUtilTest, RAMDirectoryFactoryTest, ClusterStateTest, SystemInfoHandlerTest, UUIDFieldTest, DistributedMLTComponentTest, TestRTGBase, TestTolerantSearch, ConnectionReuseTest, TestEmbeddedSolrServerConstructors, ActionThrottleTest, AliasIntegrationTest, CleanupOldIndexTest, CollectionsAPIAsyncDistributedZkTest, ConnectionManagerTest, DeleteReplicaTest, DistributedQueueTest, HttpPartitionTest, SimpleCollectionCreateDeleteTest, SolrCloudExampleTest, SolrXmlInZkTest, TestAuthenticationFramework, TestCloudDeleteByQuery, TestCloudInspectUtil, TestCloudPivotFacet, TestCollectionAPI, TestConfigSetsAPI, TestConfigSetsAPIExclusivity, TestConfigSetsAPIZkFailure, TestDistribDocBasedVersion, TestDownShardTolerantSearch, TestExclusionRuleCollectionAccess, TestLeaderElectionZkExpiry, TestLeaderInitiatedRecoveryThread, TestMiniSolrCloudClusterBase, TestMiniSolrCloudClusterSSL, TestOnReconnectListenerSupport, TestRebalanceLeaders, TestShortCircuitedRequests, TestStressLiveNodes, HdfsChaosMonkeyNothingIsSafeTest, HdfsCollectionsAPIDistributedZkTest]
   [junit4] Completed [471/573 (1!)] on J1 in 399.35s, 1 test, 1 error <<< FAILURES!

[...truncated 344 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/build.xml:757: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/build.xml:694: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/common-build.xml:534: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/lucene/common-build.xml:1477: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/lucene/common-build.xml:1033: There were test failures: 573 suites (7 ignored), 2319 tests, 1 error, 34 ignored (18 assumptions) [seed: CB8363A083754520]

Total time: 204 minutes 48 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
No prior successful build to compare, so performing full copy of artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any




[JENKINS] Lucene-Solr-NightlyTests-5.5 - Build # 20 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.5/20/

4 tests failed.
FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Timeout occured while waiting response from server at: https://127.0.0.1:34772

Stack Trace:
org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: https://127.0.0.1:34772
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:586)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.makeRequest(CollectionsAPIDistributedZkTest.java:400)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testErrorHandling(CollectionsAPIDistributedZkTest.java:516)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test(CollectionsAPIDistributedZkTest.java:179)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:996)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:971)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.net.SocketTimeoutException: Read timed out
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.read(SocketInputStream.java:152)
	at java.net.SocketInputStream.read(SocketInputStream.java:122)
	at sun.security.ssl.InputRecord.readFully(InputRecord.java:442)
	at sun.security.ssl.InputRecord.read(InputRecord.java:480)
	at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:934)
	at sun.security.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:891)
	at sun.security.ssl.AppInputStream.read(AppInputStream.java:102)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:160)
	at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:84)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:273)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
	at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
	at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)
	at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)
	at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197)
	at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)
	at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)
	at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685)
	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487)
	at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:482)
	... 47 more


FAILED:  org.apache.solr.cloud.overseer.ZkStateReaderTest.testStateFormatUpdateWithExplicitRefresh

Error Message:
Could not find collection : c1

Stack Trace:
org.apache.solr.common.SolrException: Could not find collection : c1
	at __randomizedtesting.SeedInfo.seed([AA1E0E3140F56C5F:B5A47FC69095AA9A]:0)
	at org.apache.solr.common.cloud.ClusterState.getCollection(ClusterState.java:170)
	at org.apache.solr.cloud.overseer.ZkStateReaderTest.testStateFormatUpdate(ZkStateReaderTest.java:136)
	at org.apache.solr.cloud.overseer.ZkStateReaderTest.testStateFormatUpdateWithExplicitRefresh(ZkStateReaderTest.java:42)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.handler.TestReplicationHandler.doTestStressReplication

Error Message:
timed out waiting for collection1 startAt time to exceed: Sat Jun 25 06:21:22 MST 2016

Stack Trace:
java.lang.AssertionError: timed out waiting for collection1 startAt time to exceed: Sat Jun 25 06:21:22 MST 2016
	at __randomizedtesting.SeedInfo.seed([AA1E0E3140F56C5F:71B50EF745DD05EC]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.solr.handler.TestReplicationHandler.watchCoreStartAt(TestReplicationHandler.java:1501)
	at org.apache.solr.handler.TestReplicationHandler.doTestStressReplication(TestReplicationHandler.java:853)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.test

Error Message:
Timeout occured while waiting response from server at: http://127.0.0.1:43971

Stack Trace:
org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:43971
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:586)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.makeRequest(CollectionsAPIDistributedZkTest.java:400)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testNoCollectionSpecified(CollectionsAPIDistributedZkTest.java:576)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test(CollectionsAPIDistributedZkTest.java:180)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:996)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:971)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.net.SocketTimeoutException: Read timed out
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.read(SocketInputStream.java:152)
	at java.net.SocketInputStream.read(SocketInputStream.java:122)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:160)
	at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:84)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:273)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
	at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
	at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)
	at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)
	at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197)
	at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)
	at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)
	at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685)
	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487)
	at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:482)
	... 47 more




Build Log:
[...truncated 11125 lines...]
   [junit4] Suite: org.apache.solr.cloud.overseer.ZkStateReaderTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/build/solr-core/test/J2/temp/solr.cloud.overseer.ZkStateReaderTest_AA1E0E3140F56C5F-001/init-core-data-001
   [junit4]   2> 178276 INFO  (SUITE-ZkStateReaderTest-seed#[AA1E0E3140F56C5F]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false)
   [junit4]   2> 178281 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithTimeDelayLazy-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testStateFormatUpdateWithTimeDelayLazy
   [junit4]   2> 178281 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithTimeDelayLazy-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 178297 INFO  (Thread-617) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 178297 INFO  (Thread-617) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 178436 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithTimeDelayLazy-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.ZkTestServer start zk server on port:40406
   [junit4]   2> 178436 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithTimeDelayLazy-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 178438 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithTimeDelayLazy-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 178619 INFO  (zkCallback-225-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@104f141d name:ZooKeeperConnection Watcher:127.0.0.1:40406 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 178619 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithTimeDelayLazy-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 178620 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithTimeDelayLazy-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 178632 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithTimeDelayLazy-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 178663 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithTimeDelayLazy-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 178687 INFO  (zkCallback-226-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@772d4925 name:ZooKeeperConnection Watcher:127.0.0.1:40406 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 178687 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithTimeDelayLazy-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 178687 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithTimeDelayLazy-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 178687 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithTimeDelayLazy-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 178700 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithTimeDelayLazy-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 178715 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithTimeDelayLazy-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 178727 INFO  (zkCallback-227-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@73e4ad43 name:ZooKeeperConnection Watcher:127.0.0.1:40406/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 178727 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithTimeDelayLazy-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 178727 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithTimeDelayLazy-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 178732 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithTimeDelayLazy-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 178733 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithTimeDelayLazy-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections
   [junit4]   2> 178744 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithTimeDelayLazy-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 178745 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithTimeDelayLazy-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 178746 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithTimeDelayLazy-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 178752 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithTimeDelayLazy-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 178754 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithTimeDelayLazy-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/c1
   [junit4]   2> 178764 INFO  (zkCallback-227-thread-1) [    ] o.a.s.c.c.ZkStateReader A collections change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections], has occurred - updating...
   [junit4]   2> 178794 INFO  (zkCallback-227-thread-2) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [0])
   [junit4]   2> 178819 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithTimeDelayLazy-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/c1/state.json
   [junit4]   2> 178823 INFO  (zkCallback-227-thread-2) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [0])
   [junit4]   2> 178874 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithTimeDelayLazy-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:40406 40406
   [junit4]   2> 179071 INFO  (Thread-617) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:40406 40406
   [junit4]   2> 179091 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithTimeDelayLazy-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testStateFormatUpdateWithTimeDelayLazy
   [junit4]   2> 179107 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithExplicitRefresh-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testStateFormatUpdateWithExplicitRefresh
   [junit4]   2> 179107 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithExplicitRefresh-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 179125 INFO  (Thread-618) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 179126 INFO  (Thread-618) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 179227 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithExplicitRefresh-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.ZkTestServer start zk server on port:52670
   [junit4]   2> 179227 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithExplicitRefresh-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 179239 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithExplicitRefresh-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 179246 INFO  (zkCallback-228-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4599a9a name:ZooKeeperConnection Watcher:127.0.0.1:52670 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 179246 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithExplicitRefresh-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 179246 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithExplicitRefresh-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 179248 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithExplicitRefresh-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 179269 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithExplicitRefresh-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 179455 INFO  (zkCallback-229-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@12b0d705 name:ZooKeeperConnection Watcher:127.0.0.1:52670 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 179457 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithExplicitRefresh-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 179457 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithExplicitRefresh-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 179457 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithExplicitRefresh-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 179467 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithExplicitRefresh-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 179471 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithExplicitRefresh-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 179472 INFO  (zkCallback-230-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4d6761fc name:ZooKeeperConnection Watcher:127.0.0.1:52670/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 179472 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithExplicitRefresh-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 179472 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithExplicitRefresh-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 179472 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithExplicitRefresh-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 179473 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithExplicitRefresh-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections
   [junit4]   2> 179474 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithExplicitRefresh-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 179475 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithExplicitRefresh-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 179476 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithExplicitRefresh-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 179477 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithExplicitRefresh-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 179478 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithExplicitRefresh-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.ZkStateReader addZkWatch [c1]
   [junit4]   2> 179480 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithExplicitRefresh-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.ZkStateReader Deleting data for [c1]
   [junit4]   2> 179480 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithExplicitRefresh-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/c1
   [junit4]   2> 179483 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithExplicitRefresh-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/c1/state.json
   [junit4]   2> 179486 ERROR (TEST-ZkStateReaderTest.testStateFormatUpdateWithExplicitRefresh-seed#[AA1E0E3140F56C5F]-EventThread) [    ] o.a.z.ClientCnxn Error while calling watcher 
   [junit4]   2> java.util.concurrent.RejectedExecutionException: Task org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1@4a4b3349 rejected from org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor@f1423ba[Shutting down, pool size = 3, active threads = 3, queued tasks = 0, completed tasks = 0]
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2048)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:821)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1372)
   [junit4]   2> 	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.execute(ExecutorUtil.java:214)
   [junit4]   2> 	at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:110)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$3.process(SolrZkClient.java:261)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
   [junit4]   2> 179491 INFO  (zkCallback-230-thread-3) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [0])
   [junit4]   2> 179491 INFO  (zkCallback-230-thread-2) [    ] o.a.s.c.c.ZkStateReader A collections change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections], has occurred - updating...
   [junit4]   2> 179491 WARN  (zkCallback-230-thread-3) [    ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /clusterstate.json]
   [junit4]   2> 179491 WARN  (zkCallback-230-thread-2) [    ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /collections]
   [junit4]   2> 179492 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithExplicitRefresh-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:52670 52670
   [junit4]   2> 179618 INFO  (Thread-618) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:52670 52670
   [junit4]   2> 179619 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithExplicitRefresh-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testStateFormatUpdateWithExplicitRefresh
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=ZkStateReaderTest -Dtests.method=testStateFormatUpdateWithExplicitRefresh -Dtests.seed=AA1E0E3140F56C5F -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=is-IS -Dtests.timezone=America/Blanc-Sablon -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR   0.53s J2 | ZkStateReaderTest.testStateFormatUpdateWithExplicitRefresh <<<
   [junit4]    > Throwable #1: org.apache.solr.common.SolrException: Could not find collection : c1
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([AA1E0E3140F56C5F:B5A47FC69095AA9A]:0)
   [junit4]    > 	at org.apache.solr.common.cloud.ClusterState.getCollection(ClusterState.java:170)
   [junit4]    > 	at org.apache.solr.cloud.overseer.ZkStateReaderTest.testStateFormatUpdate(ZkStateReaderTest.java:136)
   [junit4]    > 	at org.apache.solr.cloud.overseer.ZkStateReaderTest.testStateFormatUpdateWithExplicitRefresh(ZkStateReaderTest.java:42)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 179626 INFO  (TEST-ZkStateReaderTest.testExternalCollectionWatchedNotWatched-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testExternalCollectionWatchedNotWatched
   [junit4]   2> 179626 INFO  (TEST-ZkStateReaderTest.testExternalCollectionWatchedNotWatched-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 179640 INFO  (Thread-619) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 179640 INFO  (Thread-619) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 179740 INFO  (TEST-ZkStateReaderTest.testExternalCollectionWatchedNotWatched-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.ZkTestServer start zk server on port:44885
   [junit4]   2> 179740 INFO  (TEST-ZkStateReaderTest.testExternalCollectionWatchedNotWatched-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 179753 INFO  (TEST-ZkStateReaderTest.testExternalCollectionWatchedNotWatched-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 179761 INFO  (zkCallback-231-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@22dfca6 name:ZooKeeperConnection Watcher:127.0.0.1:44885 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 179761 INFO  (TEST-ZkStateReaderTest.testExternalCollectionWatchedNotWatched-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 179761 INFO  (TEST-ZkStateReaderTest.testExternalCollectionWatchedNotWatched-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 179765 INFO  (TEST-ZkStateReaderTest.testExternalCollectionWatchedNotWatched-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 179781 INFO  (TEST-ZkStateReaderTest.testExternalCollectionWatchedNotWatched-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 179799 INFO  (zkCallback-232-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2ee8f4c7 name:ZooKeeperConnection Watcher:127.0.0.1:44885 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 179799 INFO  (TEST-ZkStateReaderTest.testExternalCollectionWatchedNotWatched-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 179800 INFO  (TEST-ZkStateReaderTest.testExternalCollectionWatchedNotWatched-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 179800 INFO  (TEST-ZkStateReaderTest.testExternalCollectionWatchedNotWatched-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 179802 INFO  (TEST-ZkStateReaderTest.testExternalCollectionWatchedNotWatched-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 179819 INFO  (TEST-ZkStateReaderTest.testExternalCollectionWatchedNotWatched-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 179848 INFO  (zkCallback-233-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@37df3ab1 name:ZooKeeperConnection Watcher:127.0.0.1:44885/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 179849 INFO  (TEST-ZkStateReaderTest.testExternalCollectionWatchedNotWatched-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 179849 INFO  (TEST-ZkStateReaderTest.testExternalCollectionWatchedNotWatched-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 179849 INFO  (TEST-ZkStateReaderTest.testExternalCollectionWatchedNotWatched-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 179851 INFO  (TEST-ZkStateReaderTest.testExternalCollectionWatchedNotWatched-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections
   [junit4]   2> 179852 INFO  (TEST-ZkStateReaderTest.testExternalCollectionWatchedNotWatched-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 179853 INFO  (TEST-ZkStateReaderTest.testExternalCollectionWatchedNotWatched-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 179854 INFO  (TEST-ZkStateReaderTest.testExternalCollectionWatchedNotWatched-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 179855 INFO  (TEST-ZkStateReaderTest.testExternalCollectionWatchedNotWatched-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 179856 INFO  (TEST-ZkStateReaderTest.testExternalCollectionWatchedNotWatched-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/c1
   [junit4]   2> 179858 INFO  (TEST-ZkStateReaderTest.testExternalCollectionWatchedNotWatched-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/c1/state.json
   [junit4]   2> 179859 INFO  (TEST-ZkStateReaderTest.testExternalCollectionWatchedNotWatched-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.ZkStateReader addZkWatch [c1]
   [junit4]   2> 179859 INFO  (TEST-ZkStateReaderTest.testExternalCollectionWatchedNotWatched-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.ZkStateReader Add data for [c1] ver [0]
   [junit4]   2> 179859 INFO  (TEST-ZkStateReaderTest.testExternalCollectionWatchedNotWatched-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.ZkStateReader Removing watch for uninteresting collection [c1]
   [junit4]   2> 179861 INFO  (zkCallback-233-thread-1) [    ] o.a.s.c.c.ZkStateReader A collections change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections], has occurred - updating...
   [junit4]   2> 179861 WARN  (zkCallback-233-thread-1) [    ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /collections]
   [junit4]   2> 179866 INFO  (TEST-ZkStateReaderTest.testExternalCollectionWatchedNotWatched-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:44885 44885
   [junit4]   2> 180039 INFO  (Thread-619) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:44885 44885
   [junit4]   2> 180040 INFO  (TEST-ZkStateReaderTest.testExternalCollectionWatchedNotWatched-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testExternalCollectionWatchedNotWatched
   [junit4]   2> 180055 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithTimeDelay-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testStateFormatUpdateWithTimeDelay
   [junit4]   2> 180055 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithTimeDelay-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 180067 INFO  (Thread-620) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 180067 INFO  (Thread-620) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 180164 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithTimeDelay-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.ZkTestServer start zk server on port:56598
   [junit4]   2> 180164 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithTimeDelay-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 180177 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithTimeDelay-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 180195 INFO  (zkCallback-234-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@71c97079 name:ZooKeeperConnection Watcher:127.0.0.1:56598 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 180195 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithTimeDelay-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 180195 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithTimeDelay-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 180197 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithTimeDelay-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 180207 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithTimeDelay-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 180219 INFO  (zkCallback-235-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@32b22750 name:ZooKeeperConnection Watcher:127.0.0.1:56598 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 180219 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithTimeDelay-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 180219 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithTimeDelay-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 180219 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithTimeDelay-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 180222 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithTimeDelay-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 180232 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithTimeDelay-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 180251 INFO  (zkCallback-236-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@416e3363 name:ZooKeeperConnection Watcher:127.0.0.1:56598/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 180251 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithTimeDelay-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 180251 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithTimeDelay-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 180252 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithTimeDelay-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 180253 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithTimeDelay-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections
   [junit4]   2> 180254 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithTimeDelay-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 180255 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithTimeDelay-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 180256 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithTimeDelay-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 180257 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithTimeDelay-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 180259 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithTimeDelay-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.ZkStateReader addZkWatch [c1]
   [junit4]   2> 180259 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithTimeDelay-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.ZkStateReader Deleting data for [c1]
   [junit4]   2> 180259 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithTimeDelay-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/c1
   [junit4]   2> 180260 INFO  (zkCallback-236-thread-1) [    ] o.a.s.c.c.ZkStateReader A collections change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections], has occurred - updating...
   [junit4]   2> 180261 INFO  (zkCallback-236-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [0])
   [junit4]   2> 180262 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithTimeDelay-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/c1/state.json
   [junit4]   2> 180262 INFO  (zkCallback-236-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeCreated path:/collections/c1/state.json] for collection [c1] has occurred - updating... (live nodes size: [0])
   [junit4]   2> 180263 INFO  (zkCallback-236-thread-1) [    ] o.a.s.c.c.ZkStateReader Add data for [c1] ver [0]
   [junit4]   2> 180272 INFO  (zkCallback-236-thread-2) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [0])
   [junit4]   2> 180460 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithTimeDelay-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:56598 56598
   [junit4]   2> 180651 INFO  (Thread-620) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:56598 56598
   [junit4]   2> 180652 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithTimeDelay-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testStateFormatUpdateWithTimeDelay
   [junit4]   2> 180655 INFO  (TEST-ZkStateReaderTest.testWatchedCollectionCreation-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testWatchedCollectionCreation
   [junit4]   2> 180656 INFO  (TEST-ZkStateReaderTest.testWatchedCollectionCreation-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 180658 INFO  (Thread-621) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 180658 INFO  (Thread-621) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 180758 INFO  (TEST-ZkStateReaderTest.testWatchedCollectionCreation-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.ZkTestServer start zk server on port:55983
   [junit4]   2> 180758 INFO  (TEST-ZkStateReaderTest.testWatchedCollectionCreation-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 180783 INFO  (TEST-ZkStateReaderTest.testWatchedCollectionCreation-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 180809 INFO  (zkCallback-237-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2ad7d56d name:ZooKeeperConnection Watcher:127.0.0.1:55983 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 180809 INFO  (TEST-ZkStateReaderTest.testWatchedCollectionCreation-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 180809 INFO  (TEST-ZkStateReaderTest.testWatchedCollectionCreation-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 180811 INFO  (TEST-ZkStateReaderTest.testWatchedCollectionCreation-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 180915 INFO  (TEST-ZkStateReaderTest.testWatchedCollectionCreation-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 180955 INFO  (zkCallback-238-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@46b1461e name:ZooKeeperConnection Watcher:127.0.0.1:55983 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 180955 INFO  (TEST-ZkStateReaderTest.testWatchedCollectionCreation-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 180955 INFO  (TEST-ZkStateReaderTest.testWatchedCollectionCreation-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 180955 INFO  (TEST-ZkStateReaderTest.testWatchedCollectionCreation-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 180957 INFO  (TEST-ZkStateReaderTest.testWatchedCollectionCreation-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 180991 INFO  (TEST-ZkStateReaderTest.testWatchedCollectionCreation-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 180995 INFO  (zkCallback-239-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3aa696ef name:ZooKeeperConnection Watcher:127.0.0.1:55983/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 180995 INFO  (TEST-ZkStateReaderTest.testWatchedCollectionCreation-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 180995 INFO  (TEST-ZkStateReaderTest.testWatchedCollectionCreation-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 180996 INFO  (TEST-ZkStateReaderTest.testWatchedCollectionCreation-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 180997 INFO  (TEST-ZkStateReaderTest.testWatchedCollectionCreation-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections
   [junit4]   2> 180998 INFO  (TEST-ZkStateReaderTest.testWatchedCollectionCreation-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 180999 INFO  (TEST-ZkStateReaderTest.testWatchedCollectionCreation-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 181000 INFO  (TEST-ZkStateReaderTest.testWatchedCollectionCreation-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 181001 INFO  (TEST-ZkStateReaderTest.testWatchedCollectionCreation-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 181002 INFO  (TEST-ZkStateReaderTest.testWatchedCollectionCreation-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.ZkStateReader addZkWatch [c1]
   [junit4]   2> 181002 INFO  (TEST-ZkStateReaderTest.testWatchedCollectionCreation-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.ZkStateReader Deleting data for [c1]
   [junit4]   2> 181003 INFO  (TEST-ZkStateReaderTest.testWatchedCollectionCreation-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/c1
   [junit4]   2> 181003 INFO  (zkCallback-239-thread-1) [    ] o.a.s.c.c.ZkStateReader A collections change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections], has occurred - updating...
   [junit4]   2> 181005 INFO  (TEST-ZkStateReaderTest.testWatchedCollectionCreation-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/c1/state.json
   [junit4]   2> 181006 INFO  (zkCallback-239-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeCreated path:/collections/c1/state.json] for collection [c1] has occurred - updating... (live nodes size: [0])
   [junit4]   2> 181006 INFO  (zkCallback-239-thread-1) [    ] o.a.s.c.c.ZkStateReader Add data for [c1] ver [0]
   [junit4]   2> 181058 INFO  (TEST-ZkStateReaderTest.testWatchedCollectionCreation-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:55983 55983
   [junit4]   2> 181163 INFO  (Thread-621) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:55983 55983
   [junit4]   2> 181164 INFO  (TEST-ZkStateReaderTest.testWatchedCollectionCreation-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testWatchedCollectionCreation
   [junit4]   2> 181170 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithExplicitRefreshLazy-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testStateFormatUpdateWithExplicitRefreshLazy
   [junit4]   2> 181171 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithExplicitRefreshLazy-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 181177 INFO  (Thread-622) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 181177 INFO  (Thread-622) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 181277 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithExplicitRefreshLazy-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.ZkTestServer start zk server on port:51809
   [junit4]   2> 181277 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithExplicitRefreshLazy-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 181407 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithExplicitRefreshLazy-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 181435 INFO  (zkCallback-240-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7721e8db name:ZooKeeperConnection Watcher:127.0.0.1:51809 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 181435 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithExplicitRefreshLazy-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 181435 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithExplicitRefreshLazy-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 181438 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithExplicitRefreshLazy-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 181438 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x1558792ed1c0000, likely client has closed socket
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 181455 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithExplicitRefreshLazy-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 181479 INFO  (zkCallback-241-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4e5155e7 name:ZooKeeperConnection Watcher:127.0.0.1:51809 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 181479 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithExplicitRefreshLazy-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 181479 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithExplicitRefreshLazy-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 181479 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithExplicitRefreshLazy-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 181482 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithExplicitRefreshLazy-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 181499 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithExplicitRefreshLazy-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 181512 INFO  (zkCallback-242-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6a34b347 name:ZooKeeperConnection Watcher:127.0.0.1:51809/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 181513 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithExplicitRefreshLazy-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 181513 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithExplicitRefreshLazy-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 181514 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithExplicitRefreshLazy-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 181515 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithExplicitRefreshLazy-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections
   [junit4]   2> 181517 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithExplicitRefreshLazy-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 181518 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithExplicitRefreshLazy-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 181519 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithExplicitRefreshLazy-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 181520 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithExplicitRefreshLazy-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 181522 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithExplicitRefreshLazy-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/c1
   [junit4]   2> 181523 INFO  (zkCallback-242-thread-1) [    ] o.a.s.c.c.ZkStateReader A collections change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections], has occurred - updating...
   [junit4]   2> 181534 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithExplicitRefreshLazy-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/c1/state.json
   [junit4]   2> 181546 INFO  (zkCallback-242-thread-2) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [0])
   [junit4]   2> 181546 WARN  (zkCallback-242-thread-2) [    ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /clusterstate.json]
   [junit4]   2> 181547 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithExplicitRefreshLazy-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:51809 51809
   [junit4]   2> 181660 INFO  (Thread-622) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:51809 51809
   [junit4]   2> 181661 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithExplicitRefreshLazy-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testStateFormatUpdateWithExplicitRefreshLazy
   [junit4]   2> 181661 INFO  (SUITE-ZkStateReaderTest-seed#[AA1E0E3140F56C5F]-worker) [    ] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/build/solr-core/test/J2/temp/solr.cloud.overseer.ZkStateReaderTest_AA1E0E3140F56C5F-001
   [junit4]   2> Jun 25, 2016 12:40:01 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene54): {}, docValues:{}, sim=DefaultSimilarity, locale=is-IS, timezone=America/Blanc-Sablon
   [junit4]   2> NOTE: Linux 3.13.0-85-generic amd64/Oracle Corporation 1.7.0_80 (64-bit)/cpus=4,threads=1,free=79792208,total=340262912
   [junit4]   2> NOTE: All tests run in this JVM: [TestInitQParser, TestDistributedMissingSort, TestUtils, TestManagedSynonymFilterFactory, TestSortByMinMaxFunction, TestNoOpRegenerator, RegexBoostProcessorTest, SolrCoreCheckLockOnStartupTest, BadComponentTest, AutoCommitTest, TestScoreJoinQPScore, TestDynamicFieldResource, TestSolrDeletionPolicy1, TestPerFieldSimilarity, TestSolrConfigHandlerConcurrent, TestReversedWildcardFilterFactory, TestFieldTypeCollectionResource, CircularListTest, BasicDistributedZk2Test, ConjunctionSolrSpellCheckerTest, ZkStateReaderTest]
   [junit4] Completed [40/573 (1!)] on J2 in 3.97s, 6 tests, 1 error <<< FAILURES!

[...truncated 623 lines...]
   [junit4] Suite: org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_AA1E0E3140F56C5F-001/init-core-data-001
   [junit4]   2> 1419634 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[AA1E0E3140F56C5F]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false)
   [junit4]   2> 1419634 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[AA1E0E3140F56C5F]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   1> Formatting using clusterid: testClusterID
   [junit4]   2> 1419833 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[AA1E0E3140F56C5F]-worker) [    ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
   [junit4]   2> 1419861 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[AA1E0E3140F56C5F]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 1419912 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[AA1E0E3140F56C5F]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 1419930 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[AA1E0E3140F56C5F]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/hdfs to ./temp/Jetty_localhost_42888_hdfs____.rvezj0/webapp
   [junit4]   2> 1420218 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[AA1E0E3140F56C5F]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 1420837 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[AA1E0E3140F56C5F]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:42888
   [junit4]   2> 1421503 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[AA1E0E3140F56C5F]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 1421504 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[AA1E0E3140F56C5F]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 1421522 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[AA1E0E3140F56C5F]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_32775_datanode____.5f858j/webapp
   [junit4]   2> 1421635 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[AA1E0E3140F56C5F]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 1422057 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[AA1E0E3140F56C5F]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:32775
   [junit4]   2> 1422274 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[AA1E0E3140F56C5F]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 1422275 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[AA1E0E3140F56C5F]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 1422292 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[AA1E0E3140F56C5F]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_47946_datanode____.q6zk01/webapp
   [junit4]   2> 1422662 INFO  (IPC Server handler 5 on 32833) [    ] BlockStateChange BLOCK* processReport: from storage DS-f731e281-3927-4a29-9805-6c40acaf83a0 node DatanodeRegistration(127.0.0.1, datanodeUuid=0fea603f-66b0-4587-a70d-f73202690a7f, infoPort=32775, ipcPort=46190, storageInfo=lv=-56;cid=testClusterID;nsid=786981867;c=0), blocks: 0, hasStaleStorages: true, processing time: 0 msecs
   [junit4]   2> 1422662 INFO  (IPC Server handler 5 on 32833) [    ] BlockStateChange BLOCK* processReport: from storage DS-603bc571-e68d-4066-9574-1bff0b977fbd node DatanodeRegistration(127.0.0.1, datanodeUuid=0fea603f-66b0-4587-a70d-f73202690a7f, infoPort=32775, ipcPort=46190, storageInfo=lv=-56;cid=testClusterID;nsid=786981867;c=0), blocks: 0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 1422723 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[AA1E0E3140F56C5F]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 1423260 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[AA1E0E3140F56C5F]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:47946
   [junit4]   2> 1423924 INFO  (IPC Server handler 3 on 32833) [    ] BlockStateChange BLOCK* processReport: from storage DS-919828f6-2ff6-4175-99e7-a4d1fe6d03e6 node DatanodeRegistration(127.0.0.1, datanodeUuid=7c8c221a-4a19-44bf-af1d-6ed5b3b3ff5b, infoPort=47946, ipcPort=53198, storageInfo=lv=-56;cid=testClusterID;nsid=786981867;c=0), blocks: 0, hasStaleStorages: true, processing time: 0 msecs
   [junit4]   2> 1423924 INFO  (IPC Server handler 3 on 32833) [    ] BlockStateChange BLOCK* processReport: from storage DS-ded954fa-1e5c-4d09-bda3-447a4b5ab28a node DatanodeRegistration(127.0.0.1, datanodeUuid=7c8c221a-4a19-44bf-af1d-6ed5b3b3ff5b, infoPort=47946, ipcPort=53198, storageInfo=lv=-56;cid=testClusterID;nsid=786981867;c=0), blocks: 0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 1424250 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 1424257 INFO  (Thread-49417) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1424257 INFO  (Thread-49417) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 1424425 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.ZkTestServer start zk server on port:59611
   [junit4]   2> 1424425 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1424437 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1424497 INFO  (zkCallback-261-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1e91651c name:ZooKeeperConnection Watcher:127.0.0.1:59611 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1424498 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1424498 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1424498 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 1424500 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1424535 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1424536 INFO  (zkCallback-262-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7a2010a4 name:ZooKeeperConnection Watcher:127.0.0.1:59611/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1424536 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1424536 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1424536 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 1424538 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 1424551 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 1424552 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 1424553 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 1424553 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 1424555 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 1424555 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 1424556 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1424556 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1424557 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 1424557 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 1424558 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 1424558 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 1424559 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 1424559 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 1424560 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 1424560 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 1424561 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1424561 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1424562 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1424562 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1424563 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 1424563 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 1424564 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 1424564 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 1424684 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_AA1E0E3140F56C5F-001/control-001/cores/collection1
   [junit4]   2> 1424733 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[AA1E0E3140F56C5F]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 1424751 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[AA1E0E3140F56C5F]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@7eae363f{/,null,AVAILABLE}
   [junit4]   2> 1424751 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[AA1E0E3140F56C5F]) [    ] o.e.j.s.ServerConnector Started ServerConnector@139f23af{HTTP/1.1}{127.0.0.1:53209}
   [junit4]   2> 1424752 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[AA1E0E3140F56C5F]) [    ] o.e.j.s.Server Started @1426977ms
   [junit4]   2> 1424752 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostPort=53209, solr.data.dir=hdfs://localhost:32833/hdfs__localhost_32833__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-5.5_solr_build_solr-core_test_J0_temp_solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_AA1E0E3140F56C5F-001_tempDir-002_control_data, hostContext=/, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_AA1E0E3140F56C5F-001/control-001/cores}
   [junit4]   2> 1424752 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@7b3cb2c6
   [junit4]   2> 1424752 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_AA1E0E3140F56C5F-001/control-001'
   [junit4]   2> 1424752 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1424752 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1424752 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1424756 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1424761 INFO  (zkCallback-263-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4085de96 name:ZooKeeperConnection Watcher:127.0.0.1:59611/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1424761 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1424761 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1424880 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x15587a5e3e90002, likely client has closed socket
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 1424884 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1424885 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_AA1E0E3140F56C5F-001/control-001/solr.xml
   [junit4]   2> 1424937 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_AA1E0E3140F56C5F-001/control-001/cores
   [junit4]   2> 1424938 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.CoreContainer New CoreContainer 1901849956
   [junit4]   2> 1424938 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_AA1E0E3140F56C5F-001/control-001]
   [junit4]   2> 1424938 WARN  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_AA1E0E3140F56C5F-001/control-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_AA1E0E3140F56C5F-001/control-001/lib
   [junit4]   2> 1424938 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[AA

[...truncated too long message...]

ReplicationHandler_AA1E0E3140F56C5F-001/solr-instance-031/collection1/data/index lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@83fd91c; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
   [junit4]   2> 2720496 INFO  (coreCloseExecutor-3376-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 3
   [junit4]   2> 2720496 INFO  (coreCloseExecutor-3376-thread-1) [    x:collection1] o.a.s.c.SolrCore [collection1] Closing main searcher on request.
   [junit4]   2> 2720497 INFO  (coreCloseExecutor-3376-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing NRTCachingDirectoryFactory - 2 directories currently being tracked
   [junit4]   2> 2720497 INFO  (coreCloseExecutor-3376-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory looking to close /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_AA1E0E3140F56C5F-001/solr-instance-031/collection1/data [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_AA1E0E3140F56C5F-001/solr-instance-031/collection1/data;done=false>>]
   [junit4]   2> 2720497 INFO  (coreCloseExecutor-3376-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_AA1E0E3140F56C5F-001/solr-instance-031/collection1/data
   [junit4]   2> 2720497 INFO  (coreCloseExecutor-3376-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory looking to close /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_AA1E0E3140F56C5F-001/solr-instance-031/collection1/data/index [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_AA1E0E3140F56C5F-001/solr-instance-031/collection1/data/index;done=false>>]
   [junit4]   2> 2720497 INFO  (coreCloseExecutor-3376-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_AA1E0E3140F56C5F-001/solr-instance-031/collection1/data/index
   [junit4]   2> 2720497 INFO  (TEST-TestReplicationHandler.testRateLimitedReplication-seed#[AA1E0E3140F56C5F]) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@1c9d1e62{/solr,null,UNAVAILABLE}
   [junit4]   2> 2720501 INFO  (TEST-TestReplicationHandler.testRateLimitedReplication-seed#[AA1E0E3140F56C5F]) [    ] o.e.j.s.ServerConnector Stopped ServerConnector@7c2bb8ba{HTTP/1.1}{127.0.0.1:0}
   [junit4]   2> 2720501 INFO  (TEST-TestReplicationHandler.testRateLimitedReplication-seed#[AA1E0E3140F56C5F]) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=27695221
   [junit4]   2> 2720502 INFO  (coreCloseExecutor-3378-thread-1) [    x:collection1] o.a.s.c.SolrCore [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@74ea717f
   [junit4]   2> 2720502 INFO  (coreCloseExecutor-3378-thread-1) [    x:collection1] o.a.s.u.DirectUpdateHandler2 closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
   [junit4]   2> 2720502 INFO  (coreCloseExecutor-3378-thread-1) [    x:collection1] o.a.s.u.SolrCoreState Closing SolrCoreState
   [junit4]   2> 2720502 INFO  (coreCloseExecutor-3378-thread-1) [    x:collection1] o.a.s.u.DefaultSolrCoreState SolrCoreState ref count has reached 0 - closing IndexWriter
   [junit4]   2> 2720502 INFO  (coreCloseExecutor-3378-thread-1) [    x:collection1] o.a.s.u.DefaultSolrCoreState closing IndexWriter with IndexWriterCloser
   [junit4]   2> 2720503 INFO  (coreCloseExecutor-3378-thread-1) [    x:collection1] o.a.s.c.SolrCore [collection1] Closing main searcher on request.
   [junit4]   2> 2720503 INFO  (coreCloseExecutor-3378-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing NRTCachingDirectoryFactory - 2 directories currently being tracked
   [junit4]   2> 2720503 INFO  (coreCloseExecutor-3378-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory looking to close /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_AA1E0E3140F56C5F-001/solr-instance-032/collection1/data [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_AA1E0E3140F56C5F-001/solr-instance-032/collection1/data;done=false>>]
   [junit4]   2> 2720503 INFO  (coreCloseExecutor-3378-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_AA1E0E3140F56C5F-001/solr-instance-032/collection1/data
   [junit4]   2> 2720504 INFO  (coreCloseExecutor-3378-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory looking to close /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_AA1E0E3140F56C5F-001/solr-instance-032/collection1/data/index [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_AA1E0E3140F56C5F-001/solr-instance-032/collection1/data/index;done=false>>]
   [junit4]   2> 2720504 INFO  (coreCloseExecutor-3378-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_AA1E0E3140F56C5F-001/solr-instance-032/collection1/data/index
   [junit4]   2> 2720504 INFO  (TEST-TestReplicationHandler.testRateLimitedReplication-seed#[AA1E0E3140F56C5F]) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@61e0cb8{/solr,null,UNAVAILABLE}
   [junit4]   2> 2720505 INFO  (SUITE-TestReplicationHandler-seed#[AA1E0E3140F56C5F]-worker) [    ] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_AA1E0E3140F56C5F-001
   [junit4]   2> NOTE: test params are: codec=FastDecompressionCompressingStoredFields(storedFieldsFormat=CompressingStoredFieldsFormat(compressionMode=FAST_DECOMPRESSION, chunkSize=6, maxDocsPerChunk=728, blockSize=6), termVectorsFormat=CompressingTermVectorsFormat(compressionMode=FAST_DECOMPRESSION, chunkSize=6, blockSize=6)), sim=DefaultSimilarity, locale=ga-IE, timezone=America/Dawson_Creek
   [junit4]   2> NOTE: Linux 3.13.0-85-generic amd64/Oracle Corporation 1.7.0_80 (64-bit)/cpus=4,threads=1,free=117200792,total=455081984
   [junit4]   2> NOTE: All tests run in this JVM: [CursorMarkTest, TestCloudSchemaless, DisMaxRequestHandlerTest, TestObjectReleaseTracker, TestUniqueKeyFieldResource, TestReRankQParserPlugin, QueryEqualityTest, AtomicUpdatesTest, CachingDirectoryFactoryTest, TestRestoreCore, FileBasedSpellCheckerTest, TestSearcherReuse, TestDefaultSearchFieldResource, TestConfig, TestStressLucene, DocumentAnalysisRequestHandlerTest, TestDistributedSearch, TestFieldCollectionResource, SignatureUpdateProcessorFactoryTest, TestRandomDVFaceting, TestClassicSimilarityFactory, HdfsRecoveryZkTest, TestBulkSchemaConcurrent, TestShardHandlerFactory, TestMinMaxOnMultiValuedField, TestChildDocTransformer, EchoParamsTest, TestFastOutputStream, TestSolrXml, TestConfigReload, DistributedVersionInfoTest, HdfsCollectionsAPIDistributedZkTest, TestAnalyzedSuggestions, TestTrieFacet, TestLMDirichletSimilarityFactory, TestFoldingMultitermQuery, JsonLoaderTest, DistributedSuggestComponentTest, TestDistribIDF, TestHdfsUpdateLog, TestMaxScoreQueryParser, SolrCoreTest, PreAnalyzedFieldTest, TestNamedUpdateProcessors, TestCollapseQParserPlugin, DeleteInactiveReplicaTest, TestReplicationHandler]
   [junit4] Completed [373/573 (4!)] on J0 in 641.92s, 15 tests, 1 failure <<< FAILURES!

[...truncated 663 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/build.xml:757: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/build.xml:694: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/common-build.xml:534: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/lucene/common-build.xml:1477: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/lucene/common-build.xml:1033: There were test failures: 573 suites (7 ignored), 2319 tests, 3 errors, 1 failure, 34 ignored (18 assumptions) [seed: AA1E0E3140F56C5F]

Total time: 196 minutes 25 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
No prior successful build to compare, so performing full copy of artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any