You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2016/02/06 12:50:59 UTC

[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1089 - Still Failing

Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1089/

3 tests failed.
FAILED:  org.apache.solr.security.BasicAuthIntegrationTest.testBasics

Error Message:
Test abandoned because suite timeout was reached.

Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
	at __randomizedtesting.SeedInfo.seed([44AE6826275E59B9]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.security.BasicAuthIntegrationTest

Error Message:
Suite timeout exceeded (>= 7200000 msec).

Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
	at __randomizedtesting.SeedInfo.seed([44AE6826275E59B9]:0)


FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=12497, name=collection3, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=12497, name=collection3, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:57026: collection already exists: awholynewstresscollection_collection3_11
	at __randomizedtesting.SeedInfo.seed([44AE6826275E59B9]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:576)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:240)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:229)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1573)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1594)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:880)




Build Log:
[...truncated 11736 lines...]
   [junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_44AE6826275E59B9-001/init-core-data-001
   [junit4]   2> 1503102 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[44AE6826275E59B9]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false)
   [junit4]   2> 1503102 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[44AE6826275E59B9]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   2> 1503107 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 1503162 INFO  (Thread-5368) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1503162 INFO  (Thread-5368) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 1503261 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.c.ZkTestServer start zk server on port:38403
   [junit4]   2> 1503261 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1503310 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1503362 INFO  (zkCallback-1700-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2a731792 name:ZooKeeperConnection Watcher:127.0.0.1:38403 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1503362 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1503362 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1503362 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 1503370 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1503389 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1503390 INFO  (zkCallback-1701-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2c62347d name:ZooKeeperConnection Watcher:127.0.0.1:38403/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1503390 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1503391 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1503391 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 1503393 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 1503394 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 1503395 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 1503396 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 1503396 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 1503398 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 1503399 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 1503411 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1503411 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1503412 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 1503413 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 1503414 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 1503414 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 1503415 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 1503416 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 1503417 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 1503417 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 1503419 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1503419 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1503421 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1503421 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1503423 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 1503423 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 1503426 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 1503426 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 1503429 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1503430 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1503433 INFO  (zkCallback-1702-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@25b0179c name:ZooKeeperConnection Watcher:127.0.0.1:38403 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1503433 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1503434 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1503434 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 1503473 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1503492 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1503496 INFO  (zkCallback-1703-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@48f31c44 name:ZooKeeperConnection Watcher:127.0.0.1:38403/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1503496 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1503496 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1503497 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf2/solrconfig.xml
   [junit4]   2> 1503497 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/solrconfig.xml
   [junit4]   2> 1503505 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf2/schema.xml
   [junit4]   2> 1503505 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/schema.xml
   [junit4]   2> 1503507 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf2/enumsConfig.xml
   [junit4]   2> 1503507 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/enumsConfig.xml
   [junit4]   2> 1503515 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf2/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1503515 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1503517 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf2/stopwords.txt
   [junit4]   2> 1503517 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/stopwords.txt
   [junit4]   2> 1503519 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf2/protwords.txt
   [junit4]   2> 1503519 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/protwords.txt
   [junit4]   2> 1503533 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf2/currency.xml
   [junit4]   2> 1503534 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/currency.xml
   [junit4]   2> 1503538 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf2/open-exchange-rates.json
   [junit4]   2> 1503538 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/open-exchange-rates.json
   [junit4]   2> 1503541 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf2/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1503541 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1503543 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf2/old_synonyms.txt
   [junit4]   2> 1503543 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/old_synonyms.txt
   [junit4]   2> 1503544 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf2/synonyms.txt
   [junit4]   2> 1503545 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/synonyms.txt
   [junit4]   2> 1503546 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/elevate.xml to /configs/conf2/elevate.xml
   [junit4]   2> 1503546 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/elevate.xml
   [junit4]   2> 1504015 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_44AE6826275E59B9-001/control-001/cores/collection1
   [junit4]   2> 1504016 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 1504019 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@36a59912{/,null,AVAILABLE}
   [junit4]   2> 1504019 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.e.j.s.ServerConnector Started ServerConnector@31e488c5{HTTP/1.1}{127.0.0.1:57026}
   [junit4]   2> 1504019 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.e.j.s.Server Started @1506806ms
   [junit4]   2> 1504020 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_44AE6826275E59B9-001/tempDir-001/control/data, hostPort=57026, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_44AE6826275E59B9-001/control-001/cores}
   [junit4]   2> 1504020 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@4f6de641
   [junit4]   2> 1504020 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_44AE6826275E59B9-001/control-001'
   [junit4]   2> 1504020 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1504020 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1504020 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1504022 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1504026 INFO  (zkCallback-1704-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1ef2dec9 name:ZooKeeperConnection Watcher:127.0.0.1:38403/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1504026 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1504026 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1504029 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1504029 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_44AE6826275E59B9-001/control-001/solr.xml
   [junit4]   2> 1504056 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_44AE6826275E59B9-001/control-001/cores
   [junit4]   2> 1504056 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.c.CoreContainer New CoreContainer 2118801786
   [junit4]   2> 1504056 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_44AE6826275E59B9-001/control-001]
   [junit4]   2> 1504056 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_44AE6826275E59B9-001/control-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_44AE6826275E59B9-001/control-001/lib
   [junit4]   2> 1504056 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1504057 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1504058 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1504058 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1504058 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:38403/solr
   [junit4]   2> 1504058 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1504058 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1504061 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1504062 INFO  (zkCallback-1707-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@576f6b49 name:ZooKeeperConnection Watcher:127.0.0.1:38403 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1504063 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1504063 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1504083 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [n:127.0.0.1:57026_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1504084 INFO  (zkCallback-1708-thread-1-processing-n:127.0.0.1:57026_) [n:127.0.0.1:57026_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6af60101 name:ZooKeeperConnection Watcher:127.0.0.1:38403/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1504084 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [n:127.0.0.1:57026_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1504086 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [n:127.0.0.1:57026_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 1504087 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [n:127.0.0.1:57026_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 1504090 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [n:127.0.0.1:57026_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 1504091 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [n:127.0.0.1:57026_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 1504093 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [n:127.0.0.1:57026_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 1504094 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [n:127.0.0.1:57026_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 1504095 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [n:127.0.0.1:57026_    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 1504096 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [n:127.0.0.1:57026_    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 1504097 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [n:127.0.0.1:57026_    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 1504098 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [n:127.0.0.1:57026_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:57026_
   [junit4]   2> 1504098 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [n:127.0.0.1:57026_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:57026_
   [junit4]   2> 1504099 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [n:127.0.0.1:57026_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 1504100 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [n:127.0.0.1:57026_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 1504101 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [n:127.0.0.1:57026_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1504101 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [n:127.0.0.1:57026_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/95338420215939078-127.0.0.1:57026_-n_0000000000
   [junit4]   2> 1504102 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [n:127.0.0.1:57026_    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:57026_
   [junit4]   2> 1504102 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [n:127.0.0.1:57026_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 1504103 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [n:127.0.0.1:57026_    ] o.a.s.c.Overseer Overseer (id=95338420215939078-127.0.0.1:57026_-n_0000000000) starting
   [junit4]   2> 1504104 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [n:127.0.0.1:57026_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 1504108 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [n:127.0.0.1:57026_    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 1504109 INFO  (OverseerStateUpdate-95338420215939078-127.0.0.1:57026_-n_0000000000) [n:127.0.0.1:57026_    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 1504109 INFO  (OverseerStateUpdate-95338420215939078-127.0.0.1:57026_-n_0000000000) [n:127.0.0.1:57026_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1504113 INFO  (OverseerCollectionConfigSetProcessor-95338420215939078-127.0.0.1:57026_-n_0000000000) [n:127.0.0.1:57026_    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 1504115 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [n:127.0.0.1:57026_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1504149 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [n:127.0.0.1:57026_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1504149 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [n:127.0.0.1:57026_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1504150 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [n:127.0.0.1:57026_    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_44AE6826275E59B9-001/control-001/cores
   [junit4]   2> 1504151 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [n:127.0.0.1:57026_    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {schema=schema.xml, collection=control_collection, shard=, loadOnStartup=true, config=solrconfig.xml, coreNodeName=, dataDir=data/, configSetProperties=configsetprops.json, transient=false, name=collection1}
   [junit4]   2> 1504151 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [n:127.0.0.1:57026_    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_44AE6826275E59B9-001/control-001/cores/collection1
   [junit4]   2> 1504151 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [n:127.0.0.1:57026_    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 1504152 INFO  (coreLoadExecutor-3719-thread-1-processing-n:127.0.0.1:57026_) [n:127.0.0.1:57026_ c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 1504153 INFO  (coreLoadExecutor-3719-thread-1-processing-n:127.0.0.1:57026_) [n:127.0.0.1:57026_ c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1504154 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [n:127.0.0.1:57026_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 1504155 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [n:127.0.0.1:57026_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1504155 INFO  (OverseerStateUpdate-95338420215939078-127.0.0.1:57026_-n_0000000000) [n:127.0.0.1:57026_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "base_url":"http://127.0.0.1:57026",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "node_name":"127.0.0.1:57026_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 1504156 INFO  (coreLoadExecutor-3719-thread-1-processing-n:127.0.0.1:57026_) [n:127.0.0.1:57026_ c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 1504156 INFO  (OverseerStateUpdate-95338420215939078-127.0.0.1:57026_-n_0000000000) [n:127.0.0.1:57026_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "base_url":"http://127.0.0.1:57026",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "node_name":"127.0.0.1:57026_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 1504159 INFO  (OverseerStateUpdate-95338420215939078-127.0.0.1:57026_-n_0000000000) [n:127.0.0.1:57026_    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 1504160 INFO  (OverseerStateUpdate-95338420215939078-127.0.0.1:57026_-n_0000000000) [n:127.0.0.1:57026_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 1504161 INFO  (zkCallback-1708-thread-1-processing-n:127.0.0.1:57026_) [n:127.0.0.1:57026_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1504171 INFO  (zkCallback-1708-thread-2-processing-n:127.0.0.1:57026_) [n:127.0.0.1:57026_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1505159 INFO  (coreLoadExecutor-3719-thread-1-processing-n:127.0.0.1:57026_) [n:127.0.0.1:57026_ c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1505159 INFO  (coreLoadExecutor-3719-thread-1-processing-n:127.0.0.1:57026_) [n:127.0.0.1:57026_ c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 1505161 INFO  (coreLoadExecutor-3719-thread-1-processing-n:127.0.0.1:57026_) [n:127.0.0.1:57026_ c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 1505161 INFO  (coreLoadExecutor-3719-thread-1-processing-n:127.0.0.1:57026_) [n:127.0.0.1:57026_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/control_collection]
   [junit4]   2> 1505162 INFO  (coreLoadExecutor-3719-thread-1-processing-n:127.0.0.1:57026_) [n:127.0.0.1:57026_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/control_collection] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 1505162 INFO  (coreLoadExecutor-3719-thread-1-processing-n:127.0.0.1:57026_) [n:127.0.0.1:57026_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_44AE6826275E59B9-001/control-001/cores/collection1'
   [junit4]   2> 1505163 INFO  (coreLoadExecutor-3719-thread-1-processing-n:127.0.0.1:57026_) [n:127.0.0.1:57026_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1505163 INFO  (coreLoadExecutor-3719-thread-1-processing-n:127.0.0.1:57026_) [n:127.0.0.1:57026_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1505166 INFO  (coreLoadExecutor-3719-thread-1-processing-n:127.0.0.1:57026_) [n:127.0.0.1:57026_ c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 1505174 INFO  (coreLoadExecutor-3719-thread-1-processing-n:127.0.0.1:57026_) [n:127.0.0.1:57026_ c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 1505189 INFO  (coreLoadExecutor-3719-thread-1-processing-n:127.0.0.1:57026_) [n:127.0.0.1:57026_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.5.0
   [junit4]   2> 1505234 INFO  (coreLoadExecutor-3719-thread-1-processing-n:127.0.0.1:57026_) [n:127.0.0.1:57026_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1505266 INFO  (coreLoadExecutor-3719-thread-1-processing-n:127.0.0.1:57026_) [n:127.0.0.1:57026_ c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1505714 INFO  (coreLoadExecutor-3719-thread-1-processing-n:127.0.0.1:57026_) [n:127.0.0.1:57026_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 1505722 INFO  (coreLoadExecutor-3719-thread-1-processing-n:127.0.0.1:57026_) [n:127.0.0.1:57026_ c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 1505724 INFO  (coreLoadExecutor-3719-thread-1-processing-n:127.0.0.1:57026_) [n:127.0.0.1:57026_ c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 1505737 INFO  (coreLoadExecutor-3719-thread-1-processing-n:127.0.0.1:57026_) [n:127.0.0.1:57026_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1505741 INFO  (coreLoadExecutor-3719-thread-1-processing-n:127.0.0.1:57026_) [n:127.0.0.1:57026_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1505746 INFO  (coreLoadExecutor-3719-thread-1-processing-n:127.0.0.1:57026_) [n:127.0.0.1:57026_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1505747 INFO  (coreLoadExecutor-3719-thread-1-processing-n:127.0.0.1:57026_) [n:127.0.0.1:57026_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1505748 INFO  (coreLoadExecutor-3719-thread-1-processing-n:127.0.0.1:57026_) [n:127.0.0.1:57026_ c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 1505749 INFO  (coreLoadExecutor-3719-thread-1-processing-n:127.0.0.1:57026_) [n:127.0.0.1:57026_ c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 1505749 INFO  (coreLoadExecutor-3719-thread-1-processing-n:127.0.0.1:57026_) [n:127.0.0.1:57026_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 1505749 INFO  (coreLoadExecutor-3719-thread-1-processing-n:127.0.0.1:57026_) [n:127.0.0.1:57026_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_44AE6826275E59B9-001/control-001/cores/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_44AE6826275E59B9-001/control-001/cores/collection1/data/]
   [junit4]   2> 1505749 INFO  (coreLoadExecutor-3719-thread-1-processing-n:127.0.0.1:57026_) [n:127.0.0.1:57026_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@446da04c
   [junit4]   2> 1505750 INFO  (coreLoadExecutor-3719-thread-1-processing-n:127.0.0.1:57026_) [n:127.0.0.1:57026_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_44AE6826275E59B9-001/control-001/cores/collection1/data
   [junit4]   2> 1505750 INFO  (coreLoadExecutor-3719-thread-1-processing-n:127.0.0.1:57026_) [n:127.0.0.1:57026_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_44AE6826275E59B9-001/control-001/cores/collection1/data/index/
   [junit4]   2> 1505750 WARN  (coreLoadExecutor-3719-thread-1-processing-n:127.0.0.1:57026_) [n:127.0.0.1:57026_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_44AE6826275E59B9-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 1505750 INFO  (coreLoadExecutor-3719-thread-1-processing-n:127.0.0.1:57026_) [n:127.0.0.1:57026_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_44AE6826275E59B9-001/control-001/cores/collection1/data/index
   [junit4]   2> 1505750 INFO  (coreLoadExecutor-3719-thread-1-processing-n:127.0.0.1:57026_) [n:127.0.0.1:57026_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=30, maxMergeAtOnceExplicit=48, maxMergedSegmentMB=7.4951171875, floorSegmentMB=0.2158203125, forceMergeDeletesPctAllowed=21.076048219657785, segmentsPerTier=42.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
   [junit4]   2> 1505751 INFO  (coreLoadExecutor-3719-thread-1-processing-n:127.0.0.1:57026_) [n:127.0.0.1:57026_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(NRTCachingDirectory(RAMDirectory@1fcc5ad3 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@215ee2f4; maxCacheMB=0.9775390625 maxMergeSizeMB=0.2158203125)),segFN=segments_1,generation=1}
   [junit4]   2> 1505752 INFO  (coreLoadExecutor-3719-thread-1-processing-n:127.0.0.1:57026_) [n:127.0.0.1:57026_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 1505752 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:57026_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_44AE6826275E59B9-001/control-001/cores/collection1/data/
   [junit4]   2> 1505753 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:57026_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_44AE6826275E59B9-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 1505757 INFO  (coreLoadExecutor-3719-thread-1-processing-n:127.0.0.1:57026_) [n:127.0.0.1:57026_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 1505757 INFO  (coreLoadExecutor-3719-thread-1-processing-n:127.0.0.1:57026_) [n:127.0.0.1:57026_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 1505758 INFO  (coreLoadExecutor-3719-thread-1-processing-n:127.0.0.1:57026_) [n:127.0.0.1:57026_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 1505759 INFO  (coreLoadExecutor-3719-thread-1-processing-n:127.0.0.1:57026_) [n:127.0.0.1:57026_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1505759 INFO  (coreLoadExecutor-3719-thread-1-processing-n:127.0.0.1:57026_) [n:127.0.0.1:57026_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1505759 INFO  (coreLoadExecutor-3719-thread-1-processing-n:127.0.0.1:57026_) [n:127.0.0.1:57026_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 1505759 INFO  (coreLoadExecutor-3719-thread-1-processing-n:127.0.0.1:57026_) [n:127.0.0.1:57026_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1505760 INFO  (coreLoadExecutor-3719-thread-1-processing-n:127.0.0.1:57026_) [n:127.0.0.1:57026_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1505760 INFO  (coreLoadExecutor-3719-thread-1-processing-n:127.0.0.1:57026_) [n:127.0.0.1:57026_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 1505760 WARN  (coreLoadExecutor-3719-thread-1-processing-n:127.0.0.1:57026_) [n:127.0.0.1:57026_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
   [junit4]   2> 1505769 INFO  (coreLoadExecutor-3719-thread-1-processing-n:127.0.0.1:57026_) [n:127.0.0.1:57026_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1505773 INFO  (coreLoadExecutor-3719-thread-1-processing-n:127.0.0.1:57026_) [n:127.0.0.1:57026_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1505774 INFO  (coreLoadExecutor-3719-thread-1-processing-n:127.0.0.1:57026_) [n:127.0.0.1:57026_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1505776 INFO  (coreLoadExecutor-3719-thread-1-processing-n:127.0.0.1:57026_) [n:127.0.0.1:57026_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1505781 INFO  (coreLoadExecutor-3719-thread-1-processing-n:127.0.0.1:57026_) [n:127.0.0.1:57026_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /update/json/docs,standard,/config,/update,/admin/threads,/get,/admin/segments,/admin/ping,/update/csv,/dump,/admin/system,/admin/properties,/admin/mbeans,/admin/luke,/admin/logging,/replication,/update/json,/admin/file,/admin/plugins,/schema
   [junit4]   2> 1505782 INFO  (coreLoadExecutor-3719-thread-1-processing-n:127.0.0.1:57026_) [n:127.0.0.1:57026_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 1505783 INFO  (coreLoadExecutor-3719-thread-1-processing-n:127.0.0.1:57026_) [n:127.0.0.1:57026_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1505783 INFO  (coreLoadExecutor-3719-thread-1-processing-n:127.0.0.1:57026_) [n:127.0.0.1:57026_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1505783 INFO  (coreLoadExecutor-3719-thread-1-processing-n:127.0.0.1:57026_) [n:127.0.0.1:57026_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1505783 INFO  (coreLoadExecutor-3719-thread-1-processing-n:127.0.0.1:57026_) [n:127.0.0.1:57026_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1505784 INFO  (coreLoadExecutor-3719-thread-1-processing-n:127.0.0.1:57026_) [n:127.0.0.1:57026_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@6794457f
   [junit4]   2> 1505784 INFO  (coreLoadExecutor-3719-thread-1-processing-n:127.0.0.1:57026_) [n:127.0.0.1:57026_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(NRTCachingDirectory(RAMDirectory@1fcc5ad3 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@215ee2f4; maxCacheMB=0.9775390625 maxMergeSizeMB=0.2158203125)),segFN=segments_1,generation=1}
   [junit4]   2> 1505784 INFO  (coreLoadExecutor-3719-thread-1-processing-n:127.0.0.1:57026_) [n:127.0.0.1:57026_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 1505784 INFO  (coreLoadExecutor-3719-thread-1-processing-n:127.0.0.1:57026_) [n:127.0.0.1:57026_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@9e20bd6[collection1] main
   [junit4]   2> 1505785 INFO  (coreLoadExecutor-3719-thread-1-processing-n:127.0.0.1:57026_) [n:127.0.0.1:57026_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/control_collection]
   [junit4]   2> 1505786 INFO  (coreLoadExecutor-3719-thread-1-processing-n:127.0.0.1:57026_) [n:127.0.0.1:57026_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/control_collection] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 1505786 INFO  (coreLoadExecutor-3719-thread-1-processing-n:127.0.0.1:57026_) [n:127.0.0.1:57026_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 1505786 INFO  (coreLoadExecutor-3719-thread-1-processing-n:127.0.0.1:57026_) [n:127.0.0.1:57026_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1505786 INFO  (coreLoadExecutor-3719-thread-1-processing-n:127.0.0.1:57026_) [n:127.0.0.1:57026_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 1505786 INFO  (coreLoadExecutor-3719-thread-1-processing-n:127.0.0.1:57026_) [n:127.0.0.1:57026_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1505786 INFO  (coreLoadExecutor-3719-thread-1-processing-n:127.0.0.1:57026_) [n:127.0.0.1:57026_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 1505787 INFO  (coreLoadExecutor-3719-thread-1-processing-n:127.0.0.1:57026_) [n:127.0.0.1:57026_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1505787 INFO  (coreLoadExecutor-3719-thread-1-processing-n:127.0.0.1:57026_) [n:127.0.0.1:57026_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 1505787 INFO  (coreLoadExecutor-3719-thread-1-processing-n:127.0.0.1:57026_) [n:127.0.0.1:57026_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1505788 INFO  (searcherExecutor-3720-thread-1-processing-r:core_node1 s:shard1 x:collection1 c:control_collection n:127.0.0.1:57026_) [n:127.0.0.1:57026_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@9e20bd6[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1505788 INFO  (coreLoadExecutor-3719-thread-1-processing-n:127.0.0.1:57026_) [n:127.0.0.1:57026_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 1505788 INFO  (coreLoadExecutor-3719-thread-1-processing-n:127.0.0.1:57026_) [n:127.0.0.1:57026_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 1505788 INFO  (coreLoadExecutor-3719-thread-1-processing-n:127.0.0.1:57026_) [n:127.0.0.1:57026_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 1505788 INFO  (coreLoadExecutor-3719-thread-1-processing-n:127.0.0.1:57026_) [n:127.0.0.1:57026_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1525414726205440000
   [junit4]   2> 1505790 INFO  (coreLoadExecutor-3719-thread-1-processing-n:127.0.0.1:57026_) [n:127.0.0.1:57026_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 2.0ms to seed version buckets with highest version 1525414726205440000
   [junit4]   2> 1505790 INFO  (coreLoadExecutor-3719-thread-1-processing-n:127.0.0.1:57026_) [n:127.0.0.1:57026_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 1505791 INFO  (coreLoadExecutor-3719-thread-1-processing-n:127.0.0.1:57026_) [n:127.0.0.1:57026_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 1505791 INFO  (coreZkRegister-3713-thread-1-processing-r:core_node1 s:shard1 x:collection1 c:control_collection n:127.0.0.1:57026_) [n:127.0.0.1:57026_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:57026 collection:control_collection shard:shard1
   [junit4]   2> 1505792 INFO  (coreZkRegister-3713-thread-1-processing-r:core_node1 s:shard1 x:collection1 c:control_collection n:127.0.0.1:57026_) [n:127.0.0.1:57026_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 1505796 INFO  (coreZkRegister-3713-thread-1-processing-r:core_node1 s:shard1 x:collection1 c:control_collection n:127.0.0.1:57026_) [n:127.0.0.1:57026_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/95338420215939078-core_node1-n_0000000000
   [junit4]   2> 1505797 INFO  (coreZkRegister-3713-thread-1-processing-r:core_node1 s:shard1 x:collection1 c:control_collection n:127.0.0.1:57026_) [n:127.0.0.1:57026_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard=shard1 and weAreReplacement=false and leaderVoteWait=180000
   [junit4]   2> 1505799 INFO  (coreZkRegister-3713-thread-1-processing-r:core_node1 s:shard1 x:collection1 c:control_collection n:127.0.0.1:57026_) [n:127.0.0.1:57026_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1505799 INFO  (coreZkRegister-3713-thread-1-processing-r:core_node1 s:shard1 x:collection1 c:control_collection n:127.0.0.1:57026_) [n:127.0.0.1:57026_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1505800 INFO  (coreZkRegister-3713-thread-1-processing-r:core_node1 s:shard1 x:collection1 c:control_collection n:127.0.0.1:57026_) [n:127.0.0.1:57026_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:57026/collection1/
   [junit4]   2> 1505800 INFO  (coreZkRegister-3713-thread-1-processing-r:core_node1 s:shard1 x:collection1 c:control_collection n:127.0.0.1:57026_) [n:127.0.0.1:57026_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 1505800 INFO  (coreZkRegister-3713-thread-1-processing-r:core_node1 s:shard1 x:collection1 c:control_collection n:127.0.0.1:57026_) [n:127.0.0.1:57026_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:57026/collection1/ has no replicas
   [junit4]   2> 1505801 INFO  (OverseerStateUpdate-95338420215939078-127.0.0.1:57026_-n_0000000000) [n:127.0.0.1:57026_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 1505801 INFO  (coreZkRegister-3713-thread-1-processing-r:core_node1 s:shard1 x:collection1 c:control_collection n:127.0.0.1:57026_) [n:127.0.0.1:57026_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 1505803 INFO  (zkCallback-1708-thread-3-processing-n:127.0.0.1:57026_) [n:127.0.0.1:57026_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1505803 INFO  (zkCallback-1708-thread-2-processing-n:127.0.0.1:57026_) [n:127.0.0.1:57026_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1505806 INFO  (coreZkRegister-3713-thread-1-processing-r:core_node1 s:shard1 x:collection1 c:control_collection n:127.0.0.1:57026_) [n:127.0.0.1:57026_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 1505807 INFO  (coreZkRegister-3713-thread-1-processing-r:core_node1 s:shard1 x:collection1 c:control_collection n:127.0.0.1:57026_) [n:127.0.0.1:57026_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:57026/collection1/ shard1
   [junit4]   2> 1505808 INFO  (OverseerStateUpdate-95338420215939078-127.0.0.1:57026_-n_0000000000) [n:127.0.0.1:57026_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:57026",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 2
   [junit4]   2> 1505910 INFO  (zkCallback-1708-thread-2-processing-n:127.0.0.1:57026_) [n:127.0.0.1:57026_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1505910 INFO  (zkCallback-1708-thread-3-processing-n:127.0.0.1:57026_) [n:127.0.0.1:57026_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1505959 INFO  (coreZkRegister-3713-thread-1-processing-r:core_node1 s:shard1 x:collection1 c:control_collection n:127.0.0.1:57026_) [n:127.0.0.1:57026_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:57026/collection1/ and leader is http://127.0.0.1:57026/collection1/
   [junit4]   2> 1505959 INFO  (coreZkRegister-3713-thread-1-processing-r:core_node1 s:shard1 x:collection1 c:control_collection n:127.0.0.1:57026_) [n:127.0.0.1:57026_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:57026
   [junit4]   2> 1505959 INFO  (coreZkRegister-3713-thread-1-processing-r:core_node1 s:shard1 x:collection1 c:control_collection n:127.0.0.1:57026_) [n:127.0.0.1:57026_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1505959 INFO  (coreZkRegister-3713-thread-1-processing-r:core_node1 s:shard1 x:collection1 c:control_collection n:127.0.0.1:57026_) [n:127.0.0.1:57026_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 1505959 INFO  (coreZkRegister-3713-thread-1-processing-r:core_node1 s:shard1 x:collection1 c:control_collection n:127.0.0.1:57026_) [n:127.0.0.1:57026_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1505961 INFO  (OverseerStateUpdate-95338420215939078-127.0.0.1:57026_-n_0000000000) [n:127.0.0.1:57026_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "base_url":"http://127.0.0.1:57026",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "node_name":"127.0.0.1:57026_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "operation":"state"} current state version: 3
   [junit4]   2> 1505962 INFO  (OverseerStateUpdate-95338420215939078-127.0.0.1:57026_-n_0000000000) [n:127.0.0.1:57026_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "base_url":"http://127.0.0.1:57026",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "node_name":"127.0.0.1:57026_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 1506064 INFO  (zkCallback-1708-thread-3-processing-n:127.0.0.1:57026_) [n:127.0.0.1:57026_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1506064 INFO  (zkCallback-1708-thread-2-processing-n:127.0.0.1:57026_) [n:127.0.0.1:57026_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1506156 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1506157 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1506159 INFO  (zkCallback-1710-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6fa1dd14 name:ZooKeeperConnection Watcher:127.0.0.1:38403/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1506159 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1506159 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1506159 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1506161 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 1506269 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_44AE6826275E59B9-001/shard-1-001/cores/collection1
   [junit4]   2> 1506269 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_44AE6826275E59B9-001/shard-1-001
   [junit4]   2> 1506270 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 1506274 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@6348d702{/,null,AVAILABLE}
   [junit4]   2> 1506274 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.e.j.s.ServerConnector Started ServerConnector@431bf73a{HTTP/1.1}{127.0.0.1:45963}
   [junit4]   2> 1506274 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.e.j.s.Server Started @1509061ms
   [junit4]   2> 1506274 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solrconfig=solrconfig.xml, hostContext=/, hostPort=45963, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_44AE6826275E59B9-001/shard-1-001/cores, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_44AE6826275E59B9-001/tempDir-001/jetty1}
   [junit4]   2> 1506275 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@4f6de641
   [junit4]   2> 1506275 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_44AE6826275E59B9-001/shard-1-001'
   [junit4]   2> 1506275 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1506275 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1506275 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1506276 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1506278 INFO  (zkCallback-1711-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@16965da2 name:ZooKeeperConnection Watcher:127.0.0.1:38403/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1506278 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1506278 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1506279 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1506279 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_44AE6826275E59B9-001/shard-1-001/solr.xml
   [junit4]   2> 1506315 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_44AE6826275E59B9-001/shard-1-001/cores
   [junit4]   2> 1506315 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.c.CoreContainer New CoreContainer 1746043745
   [junit4]   2> 1506315 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_44AE6826275E59B9-001/shard-1-001]
   [junit4]   2> 1506315 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_44AE6826275E59B9-001/shard-1-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_44AE6826275E59B9-001/shard-1-001/lib
   [junit4]   2> 1506317 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1506319 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1506320 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1506320 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1506320 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:38403/solr
   [junit4]   2> 1506320 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1506320 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1506322 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1506324 INFO  (zkCallback-1714-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2629d244 name:ZooKeeperConnection Watcher:127.0.0.1:38403 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1506325 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1506325 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1506328 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [n:127.0.0.1:45963_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1506329 INFO  (zkCallback-1715-thread-1-processing-n:127.0.0.1:45963_) [n:127.0.0.1:45963_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3dcd8f72 name:ZooKeeperConnection Watcher:127.0.0.1:38403/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1506329 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [n:127.0.0.1:45963_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1506334 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [n:127.0.0.1:45963_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1507340 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [n:127.0.0.1:45963_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:45963_
   [junit4]   2> 1507341 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [n:127.0.0.1:45963_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:45963_
   [junit4]   2> 1507342 INFO  (zkCallback-1708-thread-2-processing-n:127.0.0.1:57026_) [n:127.0.0.1:57026_    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1507342 INFO  (zkCallback-1708-thread-3-processing-n:127.0.0.1:57026_) [n:127.0.0.1:57026_    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1507342 INFO  (zkCallback-1710-thread-1) [    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1507345 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [n:127.0.0.1:45963_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1507345 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [n:127.0.0.1:45963_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/95338420215939082-127.0.0.1:45963_-n_0000000001
   [junit4]   2> 1507346 INFO  (zkCallback-1715-thread-1-processing-n:127.0.0.1:45963_) [n:127.0.0.1:45963_    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1507348 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [n:127.0.0.1:45963_    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/95338420215939078-127.0.0.1:57026_-n_0000000000 to know if I could be the leader
   [junit4]   2> 1507396 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [n:127.0.0.1:45963_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1507396 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [n:127.0.0.1:45963_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1507398 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [n:127.0.0.1:45963_    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_44AE6826275E59B9-001/shard-1-001/cores
   [junit4]   2> 1507399 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [n:127.0.0.1:45963_    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {loadOnStartup=true, configSetProperties=configsetprops.json, collection=collection1, transient=false, coreNodeName=, name=collection1, config=solrconfig.xml, schema=schema.xml, dataDir=data/, shard=}
   [junit4]   2> 1507399 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [n:127.0.0.1:45963_    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_44AE6826275E59B9-001/shard-1-001/cores/collection1
   [junit4]   2> 1507399 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [n:127.0.0.1:45963_    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 1507400 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [n:127.0.0.1:45963_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 1507400 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[44AE6826275E59B9]) [n:127.0.0.1:45963_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1507400 INFO  (coreLoadExecutor-3730-thread-1-processing-n:127.0.0.1:45963_) [n:127.0.0.1:45963_ c:collection1   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 1507400 INFO  (coreLoadExecutor-3730-thread-1-processing-n:127.0.0.1:45963_) [n:127.0.0.1:45963_ c:collection1   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1507403 INFO  (OverseerStateUpdate-95338420215939078-127.0.0.1:57026_-n_0000000000) [n:127.0.0.1:57026_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "base_url":"http://127.0.0.1:45963",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "node_name":"127.0.0.1:45963_",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "operation":"state"} current state version: 4
   [junit4]   2> 1507403 INFO  (OverseerStateUpdate-95338420215939078-127.0.0.1:57026_-n_0000000000) [n:127.0.0.1:57026_    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "base_url":"http://127.0.0.1:45963",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "node_name":"127.0.0.1:45963_",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 1507403 INFO  (OverseerStateUpdate-95338420215939078-127.0.0.1:57026_-n_0000000000) [n:127.0.0.1:57026_    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 1507404 INFO  (OverseerStateUpdate-95338420215939078-127.0.0.1:57026_-n_0000000000) [n:127.0.0.1:57026_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2
   [junit4]   2> 1507404 INFO  (coreLoadExecutor-3730-thread-1-processing-n:127.0.0.1:45963_) [n:127.0.0.1:45963_ c:collection1   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 1507506 INFO  (zkCallback-1715-thread-1-processing-n:127.0.0.1:45963_) [n:127.0.0.1:45963_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1507506 INFO  (zkCallback-1710-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1507506 INFO  (zkCallback-1708-thread-2-processing-n:127.0.0.1:57026_) [n:127.0.0.1:57026_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1507506 INFO  (zkCallback-1708-thread-3-processing-n:127.0.0.1:57026_) [n:127.0.0.1:57026_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1508405 INFO  (coreLoadExecutor-3730-thread-1-processing-n:127.0.0.1:45963_) [n:127.0.0.1:45963_ c:collection1   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1508405 INFO  (coreLoadExecutor-3730-thread-1-processing-n:127.0.0.1:45963_) [n:127.0.0.1:45963_ c:collection1   x:collection1] o.a.s.c.ZkController Check for collection zkNode:collection1
   [junit4]   2> 1508405 INFO  (coreLoadExecutor-3730-thread-1-processing-n:127.0.0.1:45963_) [n:127.0.0.1:45963_ c:collection1   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 1508406 INFO  (coreLoadExecutor-3730-thread-1-processing-n:127.0.0.1:45963_) [n:127.0.0.1:45963_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/collection1]
   [junit4]   2> 1508406 INFO  (coreLoadExecutor-3730-thread-1-processing-n:127.0.0.1:45963_) [n:127.0.0.1:45963_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/collection1] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 1508406 INFO  (coreLoadExecutor-3730-thread-1-processing-n:127.0.0.1:45963_) [n:127.0.0.1:45963_ c:collection1   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_44AE6826275E59B9-001/shard-1-001/cores/collection1'
   [junit4]   2> 1508406 INFO  (coreLoadExecutor-3730-thread-1-processing-n:127.0.0.1:45963_) [n:127.0.0.1:45963_ c:collection1   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1508406 INFO  (coreLoadExecutor-3730-thread-1-processing-n:127.0.0.1:45963_) [n:127.0.0.1:45963_ c:collection1   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1508419 INFO  (coreLoadExecutor-3730-thread-1-processing-n:127.0.0.1:45963_) [n:127.0.0.1:45963_ c:

[...truncated too long message...]

NFO  (jetty-launcher-4877-thread-8) [n:127.0.0.1:59677_solr c:testSolrCloudCollection s:shard1 r:core_node3 x:testSolrCloudCollection_shard1_replica1] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@6c5906f8{/solr,null,UNAVAILABLE}
   [junit4]   2> 11788345 WARN  (TEST-BasicAuthIntegrationTest.testBasics-seed#[44AE6826275E59B9]-SendThread(127.0.0.1:40857)) [n:127.0.0.1:34190_solr    ] o.a.z.c.StaticHostProvider Unexpected exception
   [junit4]   2> java.lang.InterruptedException: sleep interrupted
   [junit4]   2> 	at java.lang.Thread.sleep(Native Method)
   [junit4]   2> 	at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:940)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1003)
   [junit4]   2> 11788345 WARN  (jetty-launcher-4877-thread-3-SendThread(127.0.0.1:40857)) [n:127.0.0.1:36130_solr    ] o.a.z.c.StaticHostProvider Unexpected exception
   [junit4]   2> java.lang.InterruptedException: sleep interrupted
   [junit4]   2> 	at java.lang.Thread.sleep(Native Method)
   [junit4]   2> 	at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:940)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1003)
   [junit4]   2> 11788345 WARN  (jetty-launcher-4877-thread-2-SendThread(127.0.0.1:40857)) [n:127.0.0.1:59677_solr    ] o.a.z.ClientCnxn Unexpected exception
   [junit4]   2> java.lang.InterruptedException: sleep interrupted
   [junit4]   2> 	at java.lang.Thread.sleep(Native Method)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:994)
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.security.BasicAuthIntegrationTest_44AE6826275E59B9-001
   [junit4]   2> 11788449 INFO  (jetty-launcher-4877-thread-9) [n:127.0.0.1:34190_solr c:testSolrCloudCollection s:shard1 r:core_node1 x:testSolrCloudCollection_shard1_replica2] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@30e86314{/solr,null,UNAVAILABLE}
   [junit4]   2> 11788449 INFO  (jetty-launcher-4877-thread-6) [n:127.0.0.1:36130_solr c:testSolrCloudCollection s:shard2 r:core_node2 x:testSolrCloudCollection_shard2_replica2] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@43bf0719{/solr,null,UNAVAILABLE}
   [junit4]   2> Feb 06, 2016 11:45:28 AM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
   [junit4]   2> INFO: All leaked threads terminated.
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene54): {range_facet_l=PostingsFormat(name=MockRandom), intDefault=PostingsFormat(name=Memory doPackFST= false), timestamp=PostingsFormat(name=MockRandom), range_facet_l_dv=Lucene50(blocksize=128), id=Lucene50(blocksize=128), range_facet_i_dv=PostingsFormat(name=MockRandom), multiDefault=PostingsFormat(name=MockRandom), _version_=PostingsFormat(name=Memory doPackFST= false)}, docValues:{timestamp=DocValuesFormat(name=Lucene54), range_facet_l_dv=DocValuesFormat(name=Direct), range_facet_i_dv=DocValuesFormat(name=Lucene54)}, sim=DefaultSimilarity, locale=ko-KR, timezone=America/Sitka
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_80 (64-bit)/cpus=4,threads=3,free=90274136,total=514850816
   [junit4]   2> NOTE: All tests run in this JVM: [TestTolerantSearch, RecoveryAfterSoftCommitTest, TestSort, StatelessScriptUpdateProcessorFactoryTest, UnloadDistributedZkTest, TestSweetSpotSimilarityFactory, TestManagedSchemaFieldResource, TestZkChroot, HighlighterConfigTest, TestFieldTypeCollectionResource, TestUseDocValuesAsStored2, TestWriterPerf, OverseerRolesTest, HdfsDirectoryFactoryTest, TestInitParams, DateMathParserTest, DateRangeFieldTest, TestDistributedMissingSort, ConnectionManagerTest, XmlUpdateRequestHandlerTest, TestOverriddenPrefixQueryForCustomFieldType, TestMaxScoreQueryParser, DeleteLastCustomShardedReplicaTest, TestXIncludeConfig, ZkStateWriterTest, BadIndexSchemaTest, TestObjectReleaseTracker, CachingDirectoryFactoryTest, HdfsRestartWhileUpdatingTest, TestDistribDocBasedVersion, SuggestComponentContextFilterQueryTest, TestMiniSolrCloudClusterBase, CurrencyFieldXmlFileTest, BJQParserTest, AssignTest, DistributedFacetPivotWhiteBoxTest, HdfsNNFailoverTest, TestDynamicLoading, TestIndexingPerformance, BasicDistributedZk2Test, TestCollapseQParserPlugin, TestNamedUpdateProcessors, CollectionsAPIAsyncDistributedZkTest, TestCloudPivotFacet, TestSchemaNameResource, TestMinMaxOnMultiValuedField, HttpPartitionTest, TestPHPSerializedResponseWriter, TestHdfsUpdateLog, ClusterStateTest, TestPerFieldSimilarity, TestElisionMultitermQuery, BitVectorTest, SaslZkACLProviderTest, TestPostingsSolrHighlighter, TestCoreDiscovery, TestAuthorizationFramework, TestTrieFacet, ScriptEngineTest, TestAnalyzeInfixSuggestions, TestConfigSetImmutable, TestCSVResponseWriter, TestLeaderElectionZkExpiry, TestCopyFieldCollectionResource, FullSolrCloudDistribCmdsTest, TestJsonRequest, TestSolrXml, CursorMarkTest, BadComponentTest, TestRecovery, AutoCommitTest, TestClassicSimilarityFactory, FieldAnalysisRequestHandlerTest, TestCharFilters, TestShardHandlerFactory, SimpleFacetsTest, UpdateParamsTest, SearchHandlerTest, LeaderElectionTest, RegexBoostProcessorTest, TestDynamicFieldResource, TestReload, DirectUpdateHandlerOptimizeTest, TestReversedWildcardFilterFactory, TestManagedSchema, TestRestManager, TestSolrDeletionPolicy1, OverseerTest, CollectionReloadTest, DistributedFacetPivotLongTailTest, CollectionsAPIDistributedZkTest, SyncSliceTest, OpenCloseCoreStressTest, LeaderElectionIntegrationTest, BasicZkTest, RecoveryZkTest, ClusterStateUpdateTest, TestRandomFaceting, ZkCLITest, TermVectorComponentDistributedTest, TestRealTimeGet, TestJoin, StatsComponentTest, DirectUpdateHandlerTest, HighlighterTest, DistributedQueryElevationComponentTest, CoreAdminHandlerTest, SuggesterWFSTTest, NoCacheHeaderTest, TestUpdate, FieldMutatingUpdateProcessorTest, QueryEqualityTest, TestRemoteStreaming, XsltUpdateRequestHandlerTest, IndexBasedSpellCheckerTest, StandardRequestHandlerTest, TestQueryTypes, RequestHandlersTest, TestSolrQueryParser, SolrPluginUtilsTest, ReturnFieldsTest, TestAnalyzedSuggestions, TestLFUCache, PreAnalyzedFieldTest, PrimUtilsTest, TestSuggestSpellingConverter, DateFieldTest, SliceStateTest, UUIDFieldTest, CircularListTest, TestRTGBase, CursorPagingTest, TestJettySolrRunner, AliasIntegrationTest, CloudExitableDirectoryReaderTest, CollectionTooManyReplicasTest, CustomCollectionTest, DeleteReplicaTest, DistributedQueueTest, DistributedVersionInfoTest, MigrateRouteKeyTest, MultiThreadedOCPTest, RollingRestartTest, TestRestoreCore, CoreAdminCreateDiscoverTest, CoreMergeIndexesAdminHandlerTest, InfoHandlerTest, SecurityConfHandlerTest, DistributedDebugComponentTest, DistributedFacetPivotLargeTest, DistributedFacetPivotSmallAdvancedTest, DistributedQueryComponentCustomSortTest, FacetPivotSmallTest, SpatialHeatmapFacetsTest, TestExpandComponent, JavabinLoaderTest, SmileWriterTest, TestIntervalFaceting, TestMacros, TestRawResponseWriter, TestSortingResponseWriter, TestManagedResource, TestManagedResourceStorage, TestFieldTypeResource, TestManagedSchemaDynamicFieldResource, TestSchemaVersionResource, TestSerializedLuceneMatchVersion, TestSolrQueryParserDefaultOperatorResource, TestSolrQueryParserResource, TestManagedSynonymFilterFactory, BadCopyFieldTest, ChangedSchemaMergeTest, SpatialRPTFieldTypeTest, TestCloudManagedSchema, TestCloudManagedSchemaConcurrent, BlockJoinFacetRandomTest, CloudMLTQParserTest, TestExactStatsCache, BasicAuthIntegrationTest]
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=BasicAuthIntegrationTest -Dtests.seed=44AE6826275E59B9 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=ko-KR -Dtests.timezone=America/Sitka -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
   [junit4] ERROR   0.00s J2 | BasicAuthIntegrationTest (suite) <<<
   [junit4]    > Throwable #1: java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([44AE6826275E59B9]:0)
   [junit4] Completed [566/566 (2!)] on J2 in 7221.88s, 1 test, 2 errors <<< FAILURES!

[...truncated 13 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:757: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:694: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:534: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1477: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1033: There were test failures: 566 suites (7 ignored), 2268 tests, 1 suite-level error, 2 errors, 33 ignored (18 assumptions) [seed: 44AE6826275E59B9]

Total time: 338 minutes 51 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1103 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1103/

3 tests failed.
FAILED:  org.apache.solr.cloud.BasicDistributedZk2Test.test

Error Message:
No live SolrServers available to handle this request:[http://127.0.0.1:57092/collection1]

Stack Trace:
org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:57092/collection1]
	at __randomizedtesting.SeedInfo.seed([F181951231137BA6:79D5AAC89FEF165E]:0)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:352)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:149)
	at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:942)
	at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:957)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.queryServer(AbstractFullDistribZkTestBase.java:1377)
	at org.apache.solr.BaseDistributedSearchTestCase.query(BaseDistributedSearchTestCase.java:609)
	at org.apache.solr.BaseDistributedSearchTestCase.query(BaseDistributedSearchTestCase.java:591)
	at org.apache.solr.BaseDistributedSearchTestCase.query(BaseDistributedSearchTestCase.java:570)
	at org.apache.solr.cloud.BasicDistributedZk2Test.brindDownShardIndexSomeDocsAndRecover(BasicDistributedZk2Test.java:279)
	at org.apache.solr.cloud.BasicDistributedZk2Test.test(BasicDistributedZk2Test.java:97)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:964)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:939)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:57092/collection1: no servers hosting shard: shard1
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:576)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:240)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:229)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	... 53 more


FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=25631, name=collection1, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=25631, name=collection1, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:55723: collection already exists: awholynewstresscollection_collection1_2
	at __randomizedtesting.SeedInfo.seed([F181951231137BA6]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:576)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:240)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:229)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1573)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1594)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:960)


FAILED:  org.apache.solr.handler.TestReplicationHandler.doTestIndexAndConfigAliasReplication

Error Message:
Index: 0, Size: 0

Stack Trace:
java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
	at __randomizedtesting.SeedInfo.seed([F181951231137BA6:6F27B4AF7FBD440]:0)
	at java.util.ArrayList.rangeCheck(ArrayList.java:635)
	at java.util.ArrayList.get(ArrayList.java:411)
	at org.apache.solr.handler.TestReplicationHandler.doTestIndexAndConfigAliasReplication(TestReplicationHandler.java:1241)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 14 lines...]
ERROR: Error fetching remote repo 'origin'
hudson.plugins.git.GitException: Failed to fetch from git://git.apache.org/lucene-solr.git
	at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:766)
	at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1022)
	at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1053)
	at hudson.scm.SCM.checkout(SCM.java:484)
	at hudson.model.AbstractProject.checkout(AbstractProject.java:1274)
	at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:609)
	at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
	at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:531)
	at hudson.model.Run.execute(Run.java:1738)
	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
	at hudson.model.ResourceController.execute(ResourceController.java:98)
	at hudson.model.Executor.run(Executor.java:381)
Caused by: hudson.plugins.git.GitException: Command "git -c core.askpass=true fetch --tags --progress git://git.apache.org/lucene-solr.git +refs/heads/*:refs/remotes/origin/*" returned status code 128:
stdout: 
stderr: fatal: read error: Connection reset by peer

	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1693)
	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:1441)
	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$300(CliGitAPIImpl.java:62)
	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$1.execute(CliGitAPIImpl.java:313)
	at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1.call(RemoteGitImpl.java:152)
	at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1.call(RemoteGitImpl.java:145)
	at hudson.remoting.UserRequest.perform(UserRequest.java:121)
	at hudson.remoting.UserRequest.perform(UserRequest.java:49)
	at hudson.remoting.Request$2.run(Request.java:326)
	at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
	at ......remote call to lucene(Native Method)
	at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1413)
	at hudson.remoting.UserResponse.retrieve(UserRequest.java:221)
	at hudson.remoting.Channel.call(Channel.java:778)
	at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.execute(RemoteGitImpl.java:145)
	at sun.reflect.GeneratedMethodAccessor533.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.invoke(RemoteGitImpl.java:131)
	at com.sun.proxy.$Proxy74.execute(Unknown Source)
	at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:764)
	... 11 more
ERROR: null
Retrying after 10 seconds
 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url git://git.apache.org/lucene-solr.git # timeout=10
Cleaning workspace
 > git rev-parse --verify HEAD # timeout=10
Resetting working tree
 > git reset --hard # timeout=10
 > git clean -fdx # timeout=10
Fetching upstream changes from git://git.apache.org/lucene-solr.git
 > git --version # timeout=10
 > git -c core.askpass=true fetch --tags --progress git://git.apache.org/lucene-solr.git +refs/heads/*:refs/remotes/origin/*
ERROR: Error fetching remote repo 'origin'
hudson.plugins.git.GitException: Failed to fetch from git://git.apache.org/lucene-solr.git
	at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:766)
	at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1022)
	at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1053)
	at hudson.scm.SCM.checkout(SCM.java:484)
	at hudson.model.AbstractProject.checkout(AbstractProject.java:1274)
	at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:609)
	at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
	at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:531)
	at hudson.model.Run.execute(Run.java:1738)
	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
	at hudson.model.ResourceController.execute(ResourceController.java:98)
	at hudson.model.Executor.run(Executor.java:381)
Caused by: hudson.plugins.git.GitException: Command "git -c core.askpass=true fetch --tags --progress git://git.apache.org/lucene-solr.git +refs/heads/*:refs/remotes/origin/*" returned status code 128:
stdout: 
stderr: fatal: read error: Connection reset by peer

	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1693)
	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:1441)
	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$300(CliGitAPIImpl.java:62)
	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$1.execute(CliGitAPIImpl.java:313)
	at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1.call(RemoteGitImpl.java:152)
	at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1.call(RemoteGitImpl.java:145)
	at hudson.remoting.UserRequest.perform(UserRequest.java:121)
	at hudson.remoting.UserRequest.perform(UserRequest.java:49)
	at hudson.remoting.Request$2.run(Request.java:326)
	at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
	at ......remote call to lucene(Native Method)
	at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1413)
	at hudson.remoting.UserResponse.retrieve(UserRequest.java:221)
	at hudson.remoting.Channel.call(Channel.java:778)
	at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.execute(RemoteGitImpl.java:145)
	at sun.reflect.GeneratedMethodAccessor533.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.invoke(RemoteGitImpl.java:131)
	at com.sun.proxy.$Proxy74.execute(Unknown Source)
	at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:764)
	... 11 more
ERROR: null
Retrying after 10 seconds
 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url git://git.apache.org/lucene-solr.git # timeout=10
Cleaning workspace
 > git rev-parse --verify HEAD # timeout=10
Resetting working tree
 > git reset --hard # timeout=10
 > git clean -fdx # timeout=10
Fetching upstream changes from git://git.apache.org/lucene-solr.git
 > git --version # timeout=10
 > git -c core.askpass=true fetch --tags --progress git://git.apache.org/lucene-solr.git +refs/heads/*:refs/remotes/origin/*
 > git rev-parse refs/remotes/origin/branch_5x^{commit} # timeout=10
 > git rev-parse refs/remotes/origin/origin/branch_5x^{commit} # timeout=10
Checking out Revision e8acc04c68ac74ca5757285581c42457100c990c (refs/remotes/origin/branch_5x)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f e8acc04c68ac74ca5757285581c42457100c990c
 > git rev-list e8acc04c68ac74ca5757285581c42457100c990c # timeout=10
No emails were triggered.
[Lucene-Solr-NightlyTests-5.x] $ /home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/bin/ant -Dtests.multiplier=2 -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt jenkins-nightly
Buildfile: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml

jenkins-nightly:

-print-java-info:
[java-info] java version "1.7.0_80"
[java-info] Java(TM) SE Runtime Environment (1.7.0_80-b15, Oracle Corporation)
[java-info] Java HotSpot(TM) 64-Bit Server VM (24.80-b11, Oracle Corporation)
[java-info] Test args: []

clean:

clean:

clean:

ivy-availability-check:

ivy-fail:

ivy-configure:
[ivy:configure] :: Apache Ivy 2.3.0 - 20130110142753 :: http://ant.apache.org/ivy/ ::
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/ivy-settings.xml

resolve-groovy:
[ivy:cachepath] :: resolving dependencies :: org.codehaus.groovy#groovy-all-caller;working
[ivy:cachepath] 	confs: [default]
[ivy:cachepath] 	found org.codehaus.groovy#groovy-all;2.4.4 in public
[ivy:cachepath] :: resolution report :: resolve 193ms :: artifacts dl 3ms
	---------------------------------------------------------------------
	|                  |            modules            ||   artifacts   |
	|       conf       | number| search|dwnlded|evicted|| number|dwnlded|
	---------------------------------------------------------------------
	|      default     |   1   |   0   |   0   |   0   ||   1   |   0   |
	---------------------------------------------------------------------

-test-with-heapdumps-enabled:
     [echo] Java HotSpot(TM) 64-Bit Server VM: Enabling heap dumps on OutOfMemoryError to dir '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/heapdumps'.
    [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/heapdumps

ivy-availability-check:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/ivy-settings.xml

resolve-groovy:

test:

ivy-availability-check:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/ivy-settings.xml

-clover.load:

resolve-groovy:

-init-totals:

test-core:

-clover.disable:

ivy-availability-check:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/ivy-settings.xml

-clover.load:

-clover.classpath:

-clover.setup:

clover:

-check-git-state:

-git-cleanroot:

-copy-git-state:
     [copy] Copying 1 file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build

git-autoclean:

resolve:

init:

compile-core:
    [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/classes/java
    [javac] Compiling 739 source files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/classes/java
    [javac] Note: Some input files use or override a deprecated API.
    [javac] Note: Recompile with -Xlint:deprecation for details.
    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/classes/java/org/apache/lucene/codecs/blocktree/package-info.class
    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/classes/java/org/apache/lucene/codecs/lucene54/package-info.class
    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/classes/java/org/apache/lucene/util/packed/package-info.class
    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/classes/java/org/apache/lucene/analysis/package-info.class
    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/classes/java/org/apache/lucene/document/package-info.class
    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/classes/java/org/apache/lucene/util/package-info.class
    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/classes/java/org/apache/lucene/analysis/tokenattributes/package-info.class
    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/classes/java/org/apache/lucene/codecs/lucene50/package-info.class
    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/classes/java/org/apache/lucene/index/package-info.class
    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/classes/java/org/apache/lucene/codecs/compressing/package-info.class
    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/classes/java/org/apache/lucene/codecs/lucene53/package-info.class
    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/classes/java/org/apache/lucene/util/automaton/package-info.class
    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/classes/java/org/apache/lucene/util/fst/package-info.class
    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/classes/java/org/apache/lucene/util/mutable/package-info.class
    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/classes/java/org/apache/lucene/search/similarities/package-info.class
    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/classes/java/org/apache/lucene/codecs/perfield/package-info.class
    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/classes/java/org/apache/lucene/store/package-info.class
    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/classes/java/org/apache/lucene/package-info.class
    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/classes/java/org/apache/lucene/search/package-info.class
    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/classes/java/org/apache/lucene/codecs/package-info.class
    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/classes/java/org/apache/lucene/search/spans/package-info.class
     [copy] Copying 3 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/classes/java

compile-test-framework:

-check-git-state:

-git-cleanroot:

-copy-git-state:

git-autoclean:

ivy-availability-check:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/ivy-settings.xml

resolve:

init:

compile-lucene-core:

-check-git-state:

-git-cleanroot:

-copy-git-state:

git-autoclean:

ivy-availability-check:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/ivy-settings.xml

resolve:

init:

-clover.disable:

-clover.load:

-clover.classpath:

-clover.setup:

clover:

compile-core:

compile-codecs:

-check-git-state:

-git-cleanroot:

-copy-git-state:

git-autoclean:

ivy-availability-check:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/ivy-settings.xml

resolve:

common.init:

compile-lucene-core:

init:

-clover.disable:

-clover.load:

-clover.classpath:

-clover.setup:

clover:

compile-core:
    [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/codecs/classes/java
    [javac] Compiling 64 source files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/codecs/classes/java
    [javac] Note: Some input files use or override a deprecated API.
    [javac] Note: Recompile with -Xlint:deprecation for details.
    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/codecs/classes/java/org/apache/lucene/codecs/blocktreeords/package-info.class
    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/codecs/classes/java/org/apache/lucene/codecs/memory/package-info.class
    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/codecs/classes/java/org/apache/lucene/codecs/simpletext/package-info.class
    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/codecs/classes/java/org/apache/lucene/codecs/blockterms/package-info.class
    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/codecs/classes/java/org/apache/lucene/codecs/bloom/package-info.class
    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/codecs/classes/java/org/apache/lucene/codecs/autoprefix/package-info.class
     [copy] Copying 3 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/codecs/classes/java

-clover.disable:

-clover.load:

-clover.classpath:

-clover.setup:

clover:

common.compile-core:
    [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/test-framework/classes/java
    [javac] Compiling 174 source files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/test-framework/classes/java
    [javac] Note: Some input files use or override a deprecated API.
    [javac] Note: Recompile with -Xlint:deprecation for details.
    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/test-framework/classes/java/org/apache/lucene/mockfile/package-info.class
    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/test-framework/classes/java/org/apache/lucene/codecs/cranky/package-info.class
    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/test-framework/classes/java/org/apache/lucene/codecs/compressing/dummy/package-info.class
    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/test-framework/classes/java/org/apache/lucene/codecs/asserting/package-info.class
    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/test-framework/classes/java/org/apache/lucene/codecs/ramonly/package-info.class
    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/test-framework/classes/java/org/apache/lucene/codecs/cheapbastard/package-info.class
    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/test-framework/classes/java/org/apache/lucene/codecs/mockrandom/package-info.class
     [copy] Copying 4 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/test-framework/classes/java

compile-core:

compile-test:
    [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/classes/test
    [javac] Compiling 429 source files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/classes/test
    [javac] Note: Some input files use or override a deprecated API.
    [javac] Note: Recompile with -Xlint:deprecation for details.
     [copy] Copying 3 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/classes/test

install-junit4-taskdef:

validate:

resolve-groovy:

-init-totals:

-test:
    [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/test
[loadresource] Do not set property tests.explicitclass as its length is 0.
    [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/test/temp
    [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/.caches/test-stats/core
   [junit4] <JUnit4> says नमस्ते! Master seed: AE8149EA276CA1C6
   [junit4] Executing 421 suites with 3 JVMs.
   [junit4] 
   [junit4] Started J1 PID(1140@localhost).
   [junit4] Started J0 PID(1142@localhost).
   [junit4] Started J2 PID(1141@localhost).
   [junit4] Suite: org.apache.lucene.search.TestBooleanCoord
   [junit4] Completed [1/421] on J2 in 2.01s, 76 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestTerms
   [junit4] Completed [2/421] on J1 in 6.75s, 8 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestOmitNorms
   [junit4] Completed [3/421] on J2 in 4.73s, 5 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestMinShouldMatch2
   [junit4] Completed [4/421] on J0 in 13.93s, 6 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.TestVersion
   [junit4] Completed [5/421] on J0 in 0.15s, 11 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestManyFields
   [junit4] Completed [6/421] on J1 in 14.15s, 3 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.similarities.TestBM25Similarity
   [junit4] Completed [7/421] on J1 in 0.12s, 3 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestUniqueTermCount
   [junit4] Completed [8/421] on J1 in 0.35s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.store.TestLockFactory
   [junit4] Completed [9/421] on J1 in 0.19s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestIntBlockPool
   [junit4] Completed [10/421] on J1 in 0.75s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.document.TestDocument
   [junit4] Completed [11/421] on J1 in 0.16s, 11 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.TestSPIClassIterator
   [junit4] Completed [12/421] on J1 in 0.09s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.store.TestTrackingDirectoryWrapper
   [junit4] IGNOR/A 0.02s J1 | TestTrackingDirectoryWrapper.testFsyncDoesntCreateNewFiles
   [junit4]    > Assumption #1: test only works for FSDirectory subclasses
   [junit4] Completed [13/421] on J1 in 1.91s, 47 tests, 1 skipped
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.TestIntroSorter
   [junit4] Completed [14/421] on J1 in 0.66s, 9 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestBoostQuery
   [junit4] Completed [15/421] on J1 in 0.11s, 4 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestTopDocsMerge
   [junit4] Completed [16/421] on J2 in 26.01s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestApproximationSearchEquivalence
   [junit4] Completed [17/421] on J0 in 20.82s, 10 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestCachingWrapperFilter
   [junit4] Completed [18/421] on J2 in 4.70s, 11 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.spans.TestSpans
   [junit4] Completed [19/421] on J2 in 1.97s, 26 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestNorms
   [junit4] Completed [20/421] on J2 in 19.19s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.document.TestFieldType
   [junit4] Completed [21/421] on J2 in 0.07s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.TestQueryBuilder
   [junit4] Completed [22/421] on J2 in 0.15s, 20 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.store.TestMmapDirectory
   [junit4] Completed [23/421] on J2 in 3.58s, 42 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.TestCharsRef
   [junit4] Completed [24/421] on J2 in 0.15s, 5 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestDocIdSet
   [junit4] Completed [25/421] on J2 in 0.08s, 3 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestDocumentWriter
   [junit4] Completed [26/421] on J2 in 1.36s, 6 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestDocCount
   [junit4] Completed [27/421] on J2 in 0.69s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.codecs.compressing.TestFastCompressionMode
   [junit4] Completed [28/421] on J2 in 2.07s, 11 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.TestRecyclingByteBlockAllocator
   [junit4] Completed [29/421] on J2 in 0.07s, 3 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestBagOfPostings
   [junit4] Completed [30/421] on J2 in 5.58s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestWildcardRandom
   [junit4] Completed [31/421] on J2 in 0.90s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.analysis.TestNumericTokenStream
   [junit4] Completed [32/421] on J2 in 0.05s, 6 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestQueryWrapperFilter
   [junit4] Completed [33/421] on J2 in 0.97s, 6 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.spans.TestSpanCollection
   [junit4] Completed [34/421] on J2 in 0.06s, 3 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestIndexSearcher
   [junit4] Completed [35/421] on J2 in 0.59s, 5 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestRegexpRandom
   [junit4] Completed [36/421] on J2 in 1.31s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.automaton.TestDeterminizeLexicon
   [junit4] Completed [37/421] on J2 in 7.07s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.analysis.tokenattributes.TestPackedTokenAttributeImpl
   [junit4] Completed [38/421] on J2 in 0.04s, 4 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestWildcard
   [junit4] Completed [39/421] on J2 in 0.27s, 8 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestFilteredQuery
   [junit4] Completed [40/421] on J2 in 3.68s, 11 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestReaderWrapperDVTypeCheck
   [junit4] Completed [41/421] on J2 in 0.13s, 1 test
   [junit4] 
   [junit4] HEARTBEAT J1 PID(1140@localhost): 2016-02-21T07:00:09, stalled for 69.7s at: TestDemoParallelLeafReader.testRandom
   [junit4] Suite: org.apache.lucene.util.fst.TestBytesStore
   [junit4] Completed [42/421] on J2 in 14.58s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.codecs.lucene50.TestLucene50SegmentInfoFormat
   [junit4] Completed [43/421] on J2 in 1.17s, 15 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestAutomatonQuery
   [junit4] Completed [44/421] on J2 in 2.45s, 8 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestTopFieldCollector
   [junit4] Completed [45/421] on J2 in 0.60s, 7 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.store.TestRateLimiter
   [junit4] Completed [46/421] on J2 in 0.34s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestFuzzyQuery
   [junit4] Completed [47/421] on J2 in 0.15s, 10 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestLiveFieldValues
   [junit4] Completed [48/421] on J2 in 4.60s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.TestExternalCodecs
   [junit4] Completed [49/421] on J2 in 2.68s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestAutomatonQueryUnicode
   [junit4] Completed [50/421] on J2 in 0.05s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestParallelCompositeReader
   [junit4] Completed [51/421] on J2 in 1.39s, 16 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestQueryCachingPolicy
   [junit4] Completed [52/421] on J2 in 0.04s, 1 test
   [junit4] 
   [junit4] HEARTBEAT J1 PID(1140@localhost): 2016-02-21T07:01:45, stalled for 60.3s at: TestDemoParallelLeafReader.testRandomMultipleSchemaGens
   [junit4] HEARTBEAT J2 PID(1141@localhost): 2016-02-21T07:01:57, stalled for 66.0s at: TestNumericDocValuesUpdates.testTonsOfUpdates
   [junit4] HEARTBEAT J0 PID(1142@localhost): 2016-02-21T07:02:33, stalled for 61.7s at: TestLucene53NormsFormat.testNCommonBig
   [junit4] HEARTBEAT J2 PID(1141@localhost): 2016-02-21T07:02:57, stalled for  126s at: TestNumericDocValuesUpdates.testTonsOfUpdates
   [junit4] HEARTBEAT J0 PID(1142@localhost): 2016-02-21T07:03:33, stalled for  122s at: TestLucene53NormsFormat.testNCommonBig
   [junit4] HEARTBEAT J1 PID(1140@localhost): 2016-02-21T07:03:45, stalled for 67.8s at: TestDemoParallelLeafReader.testRandomMultipleSchemaGensSameField
   [junit4] HEARTBEAT J2 PID(1141@localhost): 2016-02-21T07:03:57, stalled for  186s at: TestNumericDocValuesUpdates.testTonsOfUpdates
   [junit4] Suite: org.apache.lucene.codecs.lucene53.TestLucene53NormsFormat
   [junit4] IGNOR/A 0.01s J0 | TestLucene53NormsFormat.testMergeStability
   [junit4]    > Assumption #1: The MockRandom PF randomizes content on the fly, so we can't check it
   [junit4] Completed [53/421] on J0 in 307.34s, 17 tests, 1 skipped
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.TestCollectionUtil
   [junit4] Completed [54/421] on J0 in 12.37s, 4 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestTimeLimitingCollector
   [junit4] Completed [55/421] on J0 in 5.23s, 7 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestNoMergeScheduler
   [junit4] Completed [56/421] on J0 in 0.07s, 3 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestTotalHitCountCollector
   [junit4] Completed [57/421] on J0 in 0.04s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.spans.TestSpansAdvanced
   [junit4] Completed [58/421] on J0 in 0.14s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestBooleanScorer
   [junit4] Completed [59/421] on J0 in 3.72s, 5 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestDateSort
   [junit4] Completed [60/421] on J0 in 0.09s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.TestNumericUtils
   [junit4] Completed [61/421] on J0 in 2.17s, 12 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestEarlyTermination
   [junit4] Completed [62/421] on J0 in 0.65s, 1 test
   [junit4] 
   [junit4] HEARTBEAT J1 PID(1140@localhost): 2016-02-21T07:04:45, stalled for  128s at: TestDemoParallelLeafReader.testRandomMultipleSchemaGensSameField
   [junit4] Suite: org.apache.lucene.util.automaton.TestMinimize
   [junit4] Completed [63/421] on J0 in 6.79s, 3 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestPerSegmentDeletes
   [junit4] Completed [64/421] on J0 in 0.13s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.store.TestNIOFSDirectory
   [junit4] Completed [65/421] on J0 in 1.64s, 42 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestDemoParallelLeafReader
   [junit4] Completed [66/421] on J1 in 357.45s, 5 tests
   [junit4] 
   [junit4] HEARTBEAT J2 PID(1141@localhost): 2016-02-21T07:04:57, stalled for  246s at: TestNumericDocValuesUpdates.testTonsOfUpdates
   [junit4] Suite: org.apache.lucene.index.TestCodecs
   [junit4] Completed [67/421] on J1 in 0.67s, 3 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestTieredMergePolicy
   [junit4] Completed [68/421] on J0 in 13.33s, 6 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestMultiLevelSkipList
   [junit4] Completed [69/421] on J0 in 0.48s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestMultiThreadTermVectors
   [junit4] Completed [70/421] on J0 in 3.01s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestDocsAndPositions
   [junit4] Completed [71/421] on J0 in 1.87s, 6 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.store.TestBufferedChecksum
   [junit4] Completed [72/421] on J0 in 0.22s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.TestDemo
   [junit4] Completed [73/421] on J0 in 0.09s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.analysis.tokenattributes.TestCharTermAttributeImpl
   [junit4] Completed [74/421] on J0 in 1.14s, 12 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestMultiTermConstantScore
   [junit4] Completed [75/421] on J0 in 6.28s, 8 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestIndexReaderClose
   [junit4] Completed [76/421] on J0 in 0.18s, 3 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestRegexpRandom2
   [junit4] Completed [77/421] on J0 in 9.62s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestMixedCodecs
   [junit4] Completed [78/421] on J0 in 4.08s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.Test2BPagedBytes
   [junit4] IGNOR/A 0.00s J0 | Test2BPagedBytes.test
   [junit4]    > Assumption #1: 'monster' test group is disabled (@Monster(value=You must increase heap to > 2 G to run this))
   [junit4] Completed [79/421] on J0 in 0.04s, 1 test, 1 skipped
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestIndexWriterFromReader
   [junit4] Completed [80/421] on J0 in 4.01s, 12 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestIndexWriter
   [junit4] Completed [81/421] on J0 in 12.31s, 70 tests
   [junit4] 
   [junit4] HEARTBEAT J2 PID(1141@localhost): 2016-02-21T07:05:57, stalled for  306s at: TestNumericDocValuesUpdates.testTonsOfUpdates
   [junit4] HEARTBEAT J1 PID(1140@localhost): 2016-02-21T07:06:09, stalled for 71.6s at: TestGrowableByteArrayDataOutput.testWriteLargeStrings
   [junit4] Suite: org.apache.lucene.index.TestIndexWriterNRTIsCurrent
   [junit4] Completed [82/421] on J0 in 41.28s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.spans.TestSpanOrQuery
   [junit4] Completed [83/421] on J0 in 0.03s, 3 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.TestBytesRefHash
   [junit4] Completed [84/421] on J0 in 2.23s, 8 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestCustomSearcherSort
   [junit4] Completed [85/421] on J0 in 2.27s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.spans.TestNearSpansOrdered
   [junit4] Completed [86/421] on J0 in 0.45s, 18 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.automaton.TestLevenshteinAutomata
   [junit4] Completed [87/421] on J0 in 1.47s, 4 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestNumericDocValuesUpdates
   [junit4] Completed [88/421] on J2 in 371.78s, 29 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.TestBitDocIdSetBuilder
   [junit4] Completed [89/421] on J2 in 0.16s, 6 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.automaton.FiniteStringsIteratorTest
   [junit4] Completed [90/421] on J2 in 0.40s, 7 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestSortRescorer
   [junit4] Completed [91/421] on J2 in 0.94s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.FuzzyTermOnShortTermsTest
   [junit4] Completed [92/421] on J2 in 0.26s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestThreadedForceMerge
   [junit4] Completed [93/421] on J2 in 0.39s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestDocInverterPerFieldErrorInfo
   [junit4] Completed [94/421] on J2 in 0.07s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestCustomNorms
   [junit4] Completed [95/421] on J0 in 12.48s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestDoc
   [junit4] Completed [96/421] on J0 in 0.19s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.codecs.compressing.TestGrowableByteArrayDataOutput
   [junit4] Completed [97/421] on J1 in 115.81s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.TestSetOnce
   [junit4] Completed [98/421] on J1 in 0.07s, 4 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestAddIndexes
   [junit4] Completed [99/421] on J1 in 7.68s, 23 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.spans.TestSpanBoostQuery
   [junit4] Completed [100/421] on J1 in 0.03s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.analysis.tokenattributes.TestSimpleAttributeImpl
   [junit4] Completed [101/421] on J1 in 0.03s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestSloppyPhraseQuery2
   [junit4] Completed [102/421] on J0 in 18.71s, 9 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestPrefixInBooleanQuery
   [junit4] Completed [103/421] on J0 in 1.45s, 4 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestSortRandom
   [junit4] Completed [104/421] on J0 in 1.63s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.TestArrayUtil
   [junit4] Completed [105/421] on J0 in 3.83s, 13 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.TestIOUtils
   [junit4] Completed [106/421] on J0 in 0.11s, 18 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.spans.TestFilterSpans
   [junit4] Completed [107/421] on J0 in 0.02s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestOrdinalMap
   [junit4] Completed [108/421] on J0 in 0.14s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestStressNRT
   [junit4] Completed [109/421] on J0 in 2.13s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestMultiTermsEnum
   [junit4] Completed [110/421] on J0 in 0.02s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.TestVirtualMethod
   [junit4] Completed [111/421] on J0 in 0.04s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestPrefixQuery
   [junit4] Completed [112/421] on J1 in 18.16s, 3 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestDeletionPolicy
   [junit4] Completed [113/421] on J0 in 6.02s, 6 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.store.TestSleepingLockWrapper
   [junit4] Completed [114/421] on J0 in 6.30s, 6 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestMixedDocValuesUpdates
   [junit4] Completed [115/421] on J2 in 58.26s, 4 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestTermdocPerf
   [junit4] Completed [116/421] on J2 in 0.02s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestConsistentFieldNumbers
   [junit4] Completed [117/421] on J2 in 5.08s, 4 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.automaton.TestOperations
   [junit4] Completed [118/421] on J2 in 1.23s, 5 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestSegmentTermEnum
   [junit4] Completed [119/421] on J2 in 0.07s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.spans.TestSpanSearchEquivalence
   [junit4] Completed [120/421] on J1 in 46.46s, 33 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestFieldCacheRewriteMethod
   [junit4] Completed [121/421] on J1 in 7.98s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestPrefixRandom
   [junit4] Completed [122/421] on J1 in 0.83s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.automaton.TestAutomaton
   [junit4] Completed [123/421] on J1 in 6.38s, 56 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.codecs.compressing.TestFastDecompressionMode
   [junit4] Completed [124/421] on J1 in 3.77s, 11 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.spans.TestFieldMaskingSpanQuery
   [junit4] Completed [125/421] on J1 in 1.10s, 11 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestDisjunctionMaxQuery
   [junit4] Completed [126/421] on J1 in 1.26s, 12 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestNoMergePolicy
   [junit4] Completed [127/421] on J1 in 0.10s, 4 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.mutable.TestMutableValues
   [junit4] Completed [128/421] on J1 in 0.05s, 6 tests
   [junit4] 
   [junit4] HEARTBEAT J0 PID(1142@localhost): 2016-02-21T07:08:33, stalled for 65.8s at: Test2BPostings.test
   [junit4] Suite: org.apache.lucene.index.TestBinaryDocValuesUpdates
   [junit4] Completed [129/421] on J2 in 68.16s, 28 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestMaxPosition
   [junit4] Completed [130/421] on J2 in 0.04s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestSwappedIndexFiles
   [junit4] Completed [131/421] on J2 in 0.37s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestMatchNoDocsQuery
   [junit4] Completed [132/421] on J2 in 0.03s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.spans.TestSpansEnum
   [junit4] Completed [133/421] on J2 in 0.17s, 7 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.TestMergedIterator
   [junit4] Completed [134/421] on J2 in 0.40s, 25 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.document.TestField
   [junit4] Completed [135/421] on J2 in 0.08s, 19 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestAllFilesCheckIndexHeader
   [junit4] Completed [136/421] on J2 in 1.81s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.TestLSBRadixSorter
   [junit4] Completed [137/421] on J2 in 0.28s, 6 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestPhraseQuery
   [junit4] Completed [138/421] on J2 in 4.01s, 23 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.codecs.lucene50.TestLucene50StoredFieldsFormatHighCompression
   [junit4] Completed [139/421] on J1 in 37.77s, 19 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestTerm
   [junit4] Completed [140/421] on J1 in 0.03s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.spans.TestBasics
   [junit4] Completed [141/421] on J1 in 4.64s, 25 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.TestByteBlockPool
   [junit4] Completed [142/421] on J1 in 0.08s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestDocValuesScoring
   [junit4] Completed [143/421] on J1 in 0.07s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestFieldValueQuery
   [junit4] Completed [144/421] on J1 in 7.00s, 6 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.TestAssertions
   [junit4] Completed [145/421] on J1 in 0.02s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestTermVectorsReader
   [junit4] Completed [146/421] on J1 in 0.28s, 6 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestNRTReaderCleanup
   [junit4] Completed [147/421] on J1 in 0.11s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestPrefixFilter
   [junit4] Completed [148/421] on J1 in 0.07s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestLazyProxSkipping
   [junit4] Completed [149/421] on J1 in 0.64s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestConcurrentMergeScheduler
   [junit4] Completed [150/421] on J2 in 15.29s, 16 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.codecs.perfield.TestPerFieldPostingsFormat2
   [junit4] Completed [151/421] on J2 in 1.40s, 5 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.TestFilterIterator
   [junit4] Completed [152/421] on J2 in 0.04s, 8 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.store.TestMultiMMap
   [junit4] Completed [153/421] on J2 in 10.72s, 54 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestExceedMaxTermLength
   [junit4] Completed [154/421] on J2 in 0.03s, 1 test
   [junit4] 
   [junit4] HEARTBEAT J0 PID(1142@localhost): 2016-02-21T07:09:33, stalled for  126s at: Test2BPostings.test
   [junit4] Suite: org.apache.lucene.index.TestTransactions
   [junit4] Completed [155/421] on J2 in 2.23s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.TestFixedBitSet
   [junit4] Completed [156/421] on J1 in 27.11s, 25 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestRollingUpdates
   [junit4] Completed [157/421] on J2 in 11.59s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.spans.TestSpanExplanations
   [junit4] Completed [158/421] on J2 in 0.93s, 30 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestFilteredSearch
   [junit4] Completed [159/421] on J2 in 0.02s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.MultiCollectorTest
   [junit4] Completed [160/421] on J2 in 0.04s, 4 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.TestTimSorterWorstCase
   [junit4] Completed [161/421] on J1 in 43.36s, 1 test
   [junit4] 
   [junit4] HEARTBEAT J0 PID(1142@localhost): 2016-02-21T07:10:33, stalled for  186s at: Test2BPostings.test
   [junit4] Suite: org.apache.lucene.search.TestTermRangeFilter
   [junit4] Completed [162/421] on J1 in 8.32s, 3 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestStressIndexing
   [junit4] Completed [163/421] on J1 in 5.00s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestSegmentTermDocs
   [junit4] Completed [164/421] on J1 in 0.42s, 4 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestCrash
   [junit4] Completed [165/421] on J1 in 1.73s, 5 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestIndexWriterMaxDocs
   [junit4] IGNOR/A 0.00s J1 | TestIndexWriterMaxDocs.testExactlyAtTrueLimit
   [junit4]    > Assumption #1: 'monster' test group is disabled (@Monster(value=takes over two hours))
   [junit4] Completed [166/421] on J1 in 11.24s, 21 tests, 1 skipped
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestSegmentMerger
   [junit4] Completed [167/421] on J1 in 0.57s, 3 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestByteSlices
   [junit4] Completed [168/421] on J1 in 6.36s, 1 test
   [junit4] 
   [junit4] HEARTBEAT J0 PID(1142@localhost): 2016-02-21T07:11:33, stalled for  246s at: Test2BPostings.test
   [junit4] Suite: org.apache.lucene.util.fst.TestFSTs
   [junit4] Completed [169/421] on J1 in 70.35s, 20 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestReadOnlyIndex
   [junit4] Completed [170/421] on J1 in 0.13s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestAllFilesHaveChecksumFooter
   [junit4] Completed [171/421] on J1 in 1.57s, 1 test
   [junit4] 
   [junit4] HEARTBEAT J0 PID(1142@localhost): 2016-02-21T07:12:33, stalled for  306s at: Test2BPostings.test
   [junit4] Suite: org.apache.lucene.codecs.perfield.TestPerFieldDocValuesFormat
   [junit4] Completed [172/421] on J2 in 185.38s, 96 tests
   [junit4] 
   [junit4] HEARTBEAT J1 PID(1140@localhost): 2016-02-21T07:13:33, stalled for 70.9s at: TestIndexWriterExceptions.testTooManyTokens
   [junit4] HEARTBEAT J0 PID(1142@localhost): 2016-02-21T07:13:33, stalled for  366s at: Test2BPostings.test
   [junit4] Suite: org.apache.lucene.index.TestIndexWriterReader
   [junit4] Completed [173/421] on J2 in 71.68s, 23 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestIndexWriterMerging
   [junit4] Completed [174/421] on J2 in 14.92s, 6 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestNumericRangeQuery64
   [junit4] Completed [175/421] on J2 in 8.14s, 29 tests
   [junit4] 
   [junit4] HEARTBEAT J1 PID(1140@localhost): 2016-02-21T07:14:33, stalled for  131s at: TestIndexWriterExceptions.testTooManyTokens
   [junit4] HEARTBEAT J0 PID(1142@localhost): 2016-02-21T07:14:33, stalled for  426s at: Test2BPostings.test
   [junit4] HEARTBEAT J1 PID(1140@localhost): 2016-02-21T07:15:33, stalled for  191s at: TestIndexWriterExceptions.testTooManyTokens
   [junit4] HEARTBEAT J0 PID(1142@localhost): 2016-02-21T07:15:33, stalled for  486s at: Test2BPostings.test
   [junit4] HEARTBEAT J2 PID(1141@localhost): 2016-02-21T07:15:57, stalled for 69.1s at: TestPackedInts.testPackedLongValues
   [junit4] HEARTBEAT J0 PID(1142@localhost): 2016-02-21T07:16:33, stalled for  546s at: Test2BPostings.test
   [junit4] HEARTBEAT J2 PID(1141@localhost): 2016-02-21T07:16:57, stalled for  129s at: TestPackedInts.testPackedLongValues
   [junit4] Suite: org.apache.lucene.index.TestIndexWriterExceptions
   [junit4] Completed [176/421] on J1 in 294.59s, 36 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestReaderClosed
   [junit4] Completed [177/421] on J1 in 0.05s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.Test2BPostings
   [junit4] Completed [178/421] on J0 in 590.07s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.codecs.lucene50.TestBlockPostingsFormat2
   [junit4] Completed [179/421] on J0 in 4.80s, 4 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestIsCurrent
   [junit4] Completed [180/421] on J0 in 0.09s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.Test2BBinaryDocValues
   [junit4] IGNOR/A 0.00s J0 | Test2BBinaryDocValues.testFixedBinary
   [junit4]    > Assumption #1: 'monster' test group is disabled (@Monster(value=takes ~ 6 hours if the heap is 5gb))
   [junit4] IGNOR/A 0.00s J0 | Test2BBinaryDocValues.testVariableBinary
   [junit4]    > Assumption #1: 'monster' test group is disabled (@Monster(value=takes ~ 6 hours if the heap is 5gb))
   [junit4] Completed [181/421] on J0 in 0.00s, 2 tests, 2 skipped
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestParallelTermEnum
   [junit4] Completed [182/421] on J0 in 0.02s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.TestLongBitSet
   [junit4] Completed [183/421] on J0 in 12.50s, 9 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestRegexpQuery
   [junit4] Completed [184/421] on J0 in 0.12s, 7 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestShardSearching
   [junit4] Completed [185/421] on J0 in 16.34s, 1 test
   [junit4] 
   [junit4] HEARTBEAT J2 PID(1141@localhost): 2016-02-21T07:17:57, stalled for  189s at: TestPackedInts.testPackedLongValues
   [junit4] Suite: org.apache.lucene.index.TestIndexWriterCommit
   [junit4] Completed [186/421] on J0 in 9.65s, 13 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestForTooMuchCloning
   [junit4] Completed [187/421] on J0 in 0.13s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestDirectoryReader
   [junit4] Completed [188/421] on J0 in 3.05s, 24 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestPrefixCodedTerms
   [junit4] Completed [189/421] on J0 in 0.53s, 5 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestBagOfPositions
   [junit4] Completed [190/421] on J0 in 6.64s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestSnapshotDeletionPolicy
   [junit4] Completed [191/421] on J0 in 2.83s, 7 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.codecs.TestCodecLoadingDeadlock
   [junit4] Completed [192/421] on J0 in 0.35s, 1 test
   [junit4] 
   [junit4] HEARTBEAT J1 PID(1140@localhost): 2016-02-21T07:18:21, stalled for 71.6s at: TestLRUQueryCache.testRandom
   [junit4] HEARTBEAT J2 PID(1141@localhost): 2016-02-21T07:18:57, stalled for  249s at: TestPackedInts.testPackedLongValues
   [junit4] HEARTBEAT J0 PID(1142@localhost): 2016-02-21T07:19:21, stalled for 66.5s at: TestSearcherManager.testSearcherManager
   [junit4] Suite: org.apache.lucene.search.TestLRUQueryCache
   [junit4] Completed [193/421] on J1 in 140.57s, 17 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.store.TestNRTCachingDirectory
   [junit4] IGNOR/A 0.01s J1 | TestNRTCachingDirectory.testFsyncDoesntCreateNewFiles
   [junit4]    > Assumption #1: test only works for FSDirectory subclasses
   [junit4] Completed [194/421] on J1 in 9.56s, 43 tests, 1 skipped
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.TestRecyclingIntBlockAllocator
   [junit4] Completed [195/421] on J1 in 0.17s, 3 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.document.TestDateTools
   [junit4] Completed [196/421] on J1 in 0.16s, 5 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.Test2BTerms
   [junit4] IGNOR/A 0.00s J1 | Test2BTerms.test2BTerms
   [junit4]    > Assumption #1: 'monster' test group is disabled (@Monster(value=very slow, use 5g minimum heap))
   [junit4] Completed [197/421] on J1 in 0.01s, 1 test, 1 skipped
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.TestRamUsageEstimator
   [junit4] IGNOR/A 0.00s J1 | TestRamUsageEstimator.testPrintValues
   [junit4]    > Assumption #1: Specify -Dtests.verbose=true to print constants of RamUsageEstimator.
   [junit4] Completed [198/421] on J1 in 0.03s, 5 tests, 1 skipped
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestNeedsScores
   [junit4] Completed [199/421] on J1 in 0.06s, 5 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestBinaryTerms
   [junit4] Completed [200/421] on J1 in 0.10s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.document.TestBinaryDocument
   [junit4] Completed [201/421] on J1 in 0.02s, 2 tests
   [junit4] 
   [junit4] HEARTBEAT J2 PID(1141@localhost): 2016-02-21T07:19:57, stalled for  309s at: TestPackedInts.testPackedLongValues
   [junit4] HEARTBEAT J0 PID(1142@localhost): 2016-02-21T07:20:21, stalled for  127s at: TestSearcherManager.testSearcherManager
   [junit4] HEARTBEAT J1 PID(1140@localhost): 2016-02-21T07:20:45, stalled for 65.1s at: TestOfflineSorter.testThreadSafety
   [junit4] HEARTBEAT J2 PID(1141@localhost): 2016-02-21T07:20:57, stalled for  369s at: TestPackedInts.testPackedLongValues
   [junit4] HEARTBEAT J0 PID(1142@localhost): 2016-02-21T07:21:21, stalled for  187s at: TestSearcherManager.testSearcherManager
   [junit4] Suite: org.apache.lucene.util.TestOfflineSorter
   [junit4] Completed [202/421] on J1 in 112.77s, 7 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.TestAttributeSource
   [junit4] Completed [203/421] on J1 in 0.06s, 8 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestDirectoryReaderReopen
   [junit4] Completed [204/421] on J1 in 4.08s, 9 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestStressAdvance
   [junit4] Completed [205/421] on J1 in 5.82s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestOmitPositions
   [junit4] Completed [206/421] on J1 in 0.92s, 4 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.spans.TestSpanTermQuery
   [junit4] Completed [207/421] on J1 in 0.05s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestDocTermOrdsRewriteMethod
   [junit4] Completed [208/421] on J1 in 10.09s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestStressIndexing2
   [junit4] Completed [209/421] on J1 in 2.74s, 3 tests
   [junit4] 
   [junit4] HEARTBEAT J2 PID(1141@localhost): 2016-02-21T07:21:57, stalled for  429s at: TestPackedInts.testPackedLongValues
   [junit4] Suite: org.apache.lucene.index.TestIndexWriterWithThreads
   [junit4] IGNOR/A 0.04s J1 | TestIndexWriterWithThreads.testOpenTwoIndexWritersOnDifferentThreads
   [junit4]    > Assumption #1: aborting test: timeout obtaining lock
   [junit4] Completed [210/421] on J1 in 15.89s, 12 tests, 1 skipped
   [junit4] 
   [junit4] HEARTBEAT J0 PID(1142@localhost): 2016-02-21T07:22:21, stalled for  247s at: TestSearcherManager.testSearcherManager
   [junit4] Suite: org.apache.lucene.search.TestNumericRangeQuery32
   [junit4] Completed [211/421] on J1 in 16.32s, 23 tests
   [junit4] 
   [junit4] HEARTBEAT J2 PID(1141@localhost): 2016-02-21T07:22:57, stalled for  489s at: TestPackedInts.testPackedLongValues
   [junit4] Suite: org.apache.lucene.search.TestBooleanOr
   [junit4] Completed [212/421] on J1 in 47.63s, 7 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestFlushByRamOrCountsPolicy
   [junit4] Completed [213/421] on J1 in 3.57s, 5 tests
   [junit4] 
   [junit4] HEARTBEAT J0 PID(1142@localhost): 2016-02-21T07:23:21, stalled for  307s at: TestSearcherManager.testSearcherManager
   [junit4] Suite: org.apache.lucene.search.TestSloppyPhraseQuery
   [junit4] Completed [214/421] on J1 in 2.53s, 9 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.packed.TestPackedInts
   [junit4] Completed [215/421] on J2 in 551.11s, 28 tests, 2 skipped
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestIndexWriterMergePolicy
   [junit4] Completed [216/421] on J2 in 4.76s, 7 tests
   [junit4] 
   [junit4] HEARTBEAT J0 PID(1142@localhost): 2016-02-21T07:24:21, stalled for  367s at: TestSearcherManager.testSearcherManager
   [junit4] Suite: org.apache.lucene.search.TestSearcherManager
   [junit4] Completed [217/421] on J0 in 368.06s, 10 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.Test2BPositions
   [junit4] IGNOR/A 0.00s J0 | Test2BPositions.test
   [junit4]    > Assumption #1: 'monster' test group is disabled (@Monster(value=uses lots of space and takes a few minutes))
   [junit4] Completed [218/421] on J0 in 0.00s, 1 test, 1 skipped
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestSort
   [junit4] Completed [219/421] on J0 in 0.55s, 25 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestIndexFileDeleter
   [junit4] Completed [220/421] on J0 in 0.34s, 9 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.similarities.TestSimilarity2
   [junit4] Completed [221/421] on J0 in 0.23s, 7 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.TestSentinelIntSet
   [junit4] Completed [222/421] on J0 in 0.24s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestIndexWriterDelete
   [junit4] IGNOR/A 0.00s J1 | TestIndexWriterDelete.testIndexingThenDeleting
   [junit4]    > Assumption #1: This test cannot run with Memory codec
   [junit4] Completed [223/421] on J1 in 62.15s, 27 tests, 1 skipped
   [junit4] 
   [junit4] HEARTBEAT J2 PID(1141@localhost): 2016-02-21T07:24:45, stalled for 61.5s at: TestNRTThreads.testNRTThreads
   [junit4] Suite: org.apache.lucene.search.TestLRUFilterCache
   [junit4] Completed [224/421] on J0 in 47.11s, 10 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestBoolean2
   [junit4] Completed [225/421] on J0 in 13.17s, 11 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestPersistentSnapshotDeletionPolicy
   [junit4] Completed [226/421] on J0 in 2.37s, 13 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestAtomicUpdate
   [junit4] Completed [227/421] on J0 in 7.04s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestSearchWithThreads
   [junit4] Completed [228/421] on J0 in 9.73s, 1 test
   [junit4] 
   [junit4] HEARTBEAT J2 PID(1141@localhost): 2016-02-21T07:25:45, stalled for  122s at: TestNRTThreads.testNRTThreads
   [junit4] Suite: org.apache.lucene.index.TestDuelingCodecs
   [junit4] Completed [229/421] on J1 in 79.96s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestSearchAfter
   [junit4] Completed [230/421] on J0 in 1.63s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.codecs.compressing.TestHighCompressionMode
   [junit4] Completed [231/421] on J1 in 3.01s, 7 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestNeverDelete
   [junit4] Completed [232/421] on J1 in 6.54s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestLongPostings
   [junit4] Completed [233/421] on J1 in 7.79s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestIndexWriterUnicode
   [junit4] Completed [234/421] on J1 in 9.60s, 5 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestNRTReaderWithThreads
   [junit4] Completed [235/421] on J1 in 2.16s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestIndexWriterForceMerge
   [junit4] Completed [236/421] on J1 in 6.07s, 4 tests
   [junit4] 
   [junit4] HEARTBEAT J2 PID(1141@localhost): 2016-02-21T07:26:45, stalled for  182s at: TestNRTThreads.testNRTThreads
   [junit4] HEARTBEAT J0 PID(1142@localhost): 2016-02-21T07:26:57, stalled for 71.1s at: TestDocumentsWriterStallControl.testAccquireReleaseRace
   [junit4] HEARTBEAT J1 PID(1140@localhost): 2016-02-21T07:27:33, stalled for 68.4s at: TestPerFieldPostingsFormat.testRamBytesUsed
   [junit4] HEARTBEAT J2 PID(1141@localhost): 2016-02-21T07:27:45, stalled for  242s at: TestNRTThreads.testNRTThreads
   [junit4] HEARTBEAT J0 PID(1142@localhost): 2016-02-21T07:27:57, stalled for  131s at: TestDocumentsWriterStallControl.testAccquireReleaseRace
   [junit4] Suite: org.apache.lucene.codecs.perfield.TestPerFieldPostingsFormat
   [junit4] IGNOR/A 0.00s J1 | TestPerFieldPostingsFormat.testPostingsEnumReuse
   [junit4]    > Assumption #1: The MockRandom PF randomizes content on the fly, so we can't check it
   [junit4] IGNOR/A 0.00s J1 | TestPerFieldPostingsFormat.testMergeStability
   [junit4]    > Assumption #1: The MockRandom PF randomizes content on the fly, so we can't check it
   [junit4] Completed [237/421] on J1 in 116.12s, 30 tests, 2 skipped
   [junit4] 
   [junit4] Suite: org.apache.lucene.store.TestBufferedIndexInput
   [junit4] Completed [238/421] on J1 in 0.59s, 4 tests
   [junit4] 
   [junit4] HEARTBEAT J2 PID(1141@localhost): 2016-02-21T07:28:45, stalled for  302s at: TestNRTThreads.testNRTThreads
   [junit4] HEARTBEAT J0 PID(1142@localhost): 2016-02-21T07:28:57, stalled for  191s at: TestDocumentsWriterStallControl.testAccquireReleaseRace
   [junit4] Suite: org.apache.lucene.index.TestNRTThreads
   [junit4] Completed [239/421] on J2 in 348.41s, 1 test
   [junit4] 
   [junit4] HEARTBEAT J1 PID(1140@localhost): 2016-02-21T07:29:33, stalled for 66.5s at: TestTermsEnum.testVaryingTermsPerSegment
   [junit4] Suite: org.apache.lucene.util.TestWeakIdentityMap
   [junit4] Completed [240/421] on J2 in 17.34s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestSimpleExplanations
   [junit4] Completed [241/421] on J2 in 2.88s, 72 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestComplexExplanations
   [junit4] Completed [242/421] on J2 in 1.63s, 21 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestSegmentReader
   [junit4] Completed [243/421] on J2 in 0.68s, 7 tests
   [junit4] 
   [junit4] HEARTBEAT J0 PID(1142@localhost): 2016-02-21T07:29:57, stalled for  251s at: TestDocumentsWriterStallControl.testAccquireReleaseRace
   [junit4] Suite: org.apache.lucene.util.TestPagedBytes
   [junit4] Completed [244/421] on J2 in 4.66s, 4 tests, 1 skipped
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestMultiDocValues
   [junit4] Completed [245/421] on J2 in 19.95s, 8 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.analysis.TestToken
   [junit4] Completed [246/421] on J2 in 0.03s, 5 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestFieldsReader
   [junit4] Completed [247/421] on J2 in 0.30s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestSimpleSearchEquivalence
   [junit4] Completed [248/421] on J2 in 9.77s, 16 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestDocValuesIndexing
   [junit4] Completed [249/421] on J2 in 0.53s, 34 tests
   [junit4] 
   [junit4] HEARTBEAT J1 PID(1140@localhost): 2016-02-21T07:30:33, stalled for  126s at: TestTermsEnum.testVaryingTermsPerSegment
   [junit4] Suite: org.apache.lucene.search.similarities.TestSimilarityBase
   [junit4] Completed [250/421] on J2 in 2.28s, 32 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.store.TestHugeRamFile
   [junit4] Completed [251/421] on J2 in 7.87s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestBooleanMinShouldMatch
   [junit4] Completed [252/421] on J2 in 8.47s, 17 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestTermsEnum2
   [junit4] Completed [253/421] on J2 in 2.17s, 4 tests
   [junit4] 
   [junit4] HEARTBEAT J0 PID(1142@localhost): 2016-02-21T07:30:57, stalled for  311s at: TestDocumentsWriterStallControl.testAccquireReleaseRace
   [junit4] Suite: org.apache.lucene.index.TestIndexWriterOnDiskFull
   [junit4] Completed [254/421] on J2 in 40.11s, 4 tests
   [junit4] 
   [junit4] HEARTBEAT J1 PID(1140@localhost): 2016-02-21T07:31:33, stalled for  186s at: TestTermsEnum.testVaryingTermsPerSegment
   [junit4] Suite: org.apache.lucene.index.TestPostingsOffsets
   [junit4] Completed [255/421] on J2 in 1.60s, 13 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestTransactionRollback
   [junit4] Completed [256/421] on J2 in 0.41s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestFlex
   [junit4] Completed [257/421] on J2 in 0.49s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.automaton.TestUTF32ToUTF8
   [junit4] Completed [258/421] on J2 in 6.32s, 6 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.store.TestRAMDirectory
   [junit4] IGNOR/A 0.01s J2 | TestRAMDirectory.testFsyncDoesntCreateNewFiles
   [junit4]    > Assumption #1: test only works for FSDirectory subclasses
   [junit4] Completed [259/421] on J2 in 1.17s, 46 tests, 1 skipped
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestMultiPhraseQuery
   [junit4] Completed [260/421] on J2 in 0.24s, 17 tests, 1 skipped
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.automaton.TestDeterminism
   [junit4] Completed [261/421] on J2 in 1.03s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.TestMergeSchedulerExternal
   [junit4] Completed [262/421] on J2 in 0.40s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestTermVectorsWriter
   [junit4] Completed [263/421] on J2 in 0.39s, 15 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestParallelLeafReader
   [junit4] Completed [264/421] on J2 in 0.19s, 7 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestIndexInput
   [junit4] Completed [265/421] on J2 in 0.91s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestForceMergeForever
   [junit4] Completed [266/421] on J2 in 0.61s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestOmitTf
   [junit4] Completed [267/421] on J2 in 0.82s, 6 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestSameScoresWithThreads
   [junit4] Completed [268/421] on J2 in 1.60s, 1 test
   [junit4] 
   [junit4] HEARTBEAT J0 PID(1142@localhost): 2016-02-21T07:31:57, stalled for  371s at: TestDocumentsWriterStallControl.testAccquireReleaseRace
   [junit4] Suite: org.apache.lucene.index.TestMultiFields
   [junit4] Completed [269/421] on J2 in 10.51s, 3 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestTermRangeQuery
   [junit4] Completed [270/421] on J2 in 0.29s, 7 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestDocumentsWriterDeleteQueue
   [junit4] Completed [271/421] on J2 in 0.21s, 5 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestCrashCausesCorruptIndex
   [junit4] Completed [272/421] on J2 in 0.05s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestTermsEnum
   [junit4] Completed [273/421] on J1 in 247.15s, 11 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestPayloads
   [junit4] Completed [274/421] on J1 in 0.26s, 7 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestNoDeletionPolicy
   [junit4] Completed [275/421] on J1 in 0.16s, 4 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestSimpleExplanationsOfNonMatches
   [junit4] Completed [276/421] on J1 in 0.24s, 72 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.spans.TestSpansAdvanced2
   [junit4] Completed [277/421] on J1 in 0.33s, 4 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestMultiValuedNumericRangeQuery
   [junit4] Completed [278/421] on J1 in 2.14s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestTopDocsCollector
   [junit4] Completed [279/421] on J1 in 0.09s, 8 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestBooleanQuery
   [junit4] Completed [280/421] on J1 in 0.40s, 25 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestComplexExplanationsOfNonMatches
   [junit4] Completed [281/421] on J1 in 0.18s, 21 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.TestUnicodeUtil
   [junit4] Completed [282/421] on J1 in 0.99s, 5 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestScorerPerf
   [junit4] Completed [283/421] on J2 in 30.24s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestParallelReaderEmptyIndex
   [junit4] Completed [284/421] on J2 in 0.09s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestAllFilesHaveCodecHeader
   [junit4] Completed [285/421] on J1 in 1.39s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestTermScorer
   [junit4] Completed [286/421] on J1 in 0.13s, 4 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestMaxTermFrequency
   [junit4] Completed [287/421] on J2 in 1.85s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.store.TestFileSwitchDirectory
   [junit4] IGNOR/A 0.01s J1 | TestFileSwitchDirectory.testFsyncDoesntCreateNewFiles
   [junit4]    > Assumption #1: test only works for FSDirectory subclasses
   [junit4] Completed [288/421] on J1 in 1.31s, 43 tests, 1 skipped
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestSumDocFreq
   [junit4] Completed [289/421] on J2 in 1.04s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.store.TestDirectory
   [junit4] Completed [290/421] on J2 in 0.12s, 3 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestIndexableField
   [junit4] Completed [291/421] on J1 in 1.18s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestPayloadsOnVectors
   [junit4] Completed [292/421] on J1 in 0.05s, 3 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestCheckIndex
   [junit4] Completed [293/421] on J2 in 0.53s, 5 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestFieldValueFilter
   [junit4] Completed [294/421] on J1 in 0.31s, 3 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.spans.TestSpanMultiTermQueryWrapper
   [junit4] Completed [295/421] on J2 in 0.17s, 9 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestBooleanQueryVisitSubscorers
   [junit4] Completed [296/421] on J1 in 0.11s, 5 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.TestSearchForDuplicates
   [junit4] Completed [297/421] on J2 in 0.24s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestSizeBoundedForceMerge
   [junit4] Completed [298/421] on J1 in 0.12s, 11 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.automaton.TestCompiledAutomaton
   [junit4] Completed [299/421] on J2 in 0.22s, 6 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestConstantScoreQuery
   [junit4] Completed [300/421] on J1 in 0.07s, 6 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestMatchAllDocsQuery
   [junit4] Completed [301/421] on J2 in 0.02s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestElevationComparator
   [junit4] Completed [302/421] on J1 in 0.08s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestSubScorerFreqs
   [junit4] Completed [303/421] on J2 in 0.05s, 3 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestIndexWriterConfig
   [junit4] Completed [304/421] on J1 in 0.12s, 8 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestConjunctions
   [junit4] Completed [305/421] on J2 in 0.03s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestMultiTermQueryRewrites
   [junit4] Completed [306/421] on J1 in 0.05s, 3 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestPositionIncrement
   [junit4] Completed [307/421] on J2 in 0.08s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestSimilarity
   [junit4] Completed [308/421] on J1 in 0.02s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.spans.TestSpanExplanationsOfNonMatches
   [junit4] Completed [309/421] on J2 in 0.10s, 30 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.TestRollingBuffer
   [junit4] Completed [310/421] on J1 in 0.21s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.TestSmallFloat
   [junit4] Completed [311/421] on J2 in 0.19s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestSimilarityProvider
   [junit4] Completed [312/421] on J1 in 0.02s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestFilterLeafReader
   [junit4] Completed [313/421] on J2 in 0.04s, 3 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.TestSearch
   [junit4] Completed [314/421] on J1 in 0.15s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.analysis.TestCachingTokenFilter
   [junit4] Completed [315/421] on J2 in 0.03s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestDateFilter
   [junit4] Completed [316/421] on J1 in 0.04s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestSameTokenSamePosition
   [junit4] Completed [317/421] on J1 in 0.04s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestCachingCollector
   [junit4] Completed [318/421] on J1 in 0.05s, 4 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.TestPriorityQueue
   [junit4] Completed [319/421] on J2 in 0.57s, 7 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.spans.TestSpanFirstQuery
   [junit4] Completed [320/421] on J1 in 0.01s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.TestNamedSPILoader
   [junit4] Completed [321/421] on J2 in 0.02s, 3 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestIndexWriterLockRelease
   [junit4] Completed [322/421] on J1 in 0.06s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestPhrasePrefixQuery
   [junit4] Completed [323/421] on J2 in 0.02s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.TestBytesRef
   [junit4] Completed [324/421] on J1 in 0.02s, 3 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.analysis.TestCharFilter
   [junit4] Completed [325/421] on J2 in 0.09s, 4 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestRollback
   [junit4] Completed [326/421] on J1 in 0.02s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestTwoPhaseCommitTool
   [junit4] Completed [327/421] on J2 in 0.04s, 3 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestNot
   [junit4] Completed [328/421] on J1 in 0.02s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.TestCloseableThreadLocal
   [junit4] Completed [329/421] on J2 in 0.02s, 3 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestIndexWriterOnJRECrash
   [junit4] Completed [330/421] on J1 in 5.06s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestCodecHoldsOpenFiles
   [junit4] Completed [331/421] on J1 in 0.36s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestDocBoost
   [junit4] Completed [332/421] on J1 in 0.02s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestNGramPhraseQuery
   [junit4] Completed [333/421] on J1 in 0.03s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestScoreCachingWrappingScorer
   [junit4] Completed [334/421] on J1 in 0.01s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.TestIntsRef
   [junit4] Completed [335/421] on J1 in 0.04s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestIndexCommit
   [junit4] Completed [336/421] on J1 in 0.01s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.Test2BPostingsBytes
   [junit4] IGNOR/A 0.00s J1 | Test2BPostingsBytes.test
   [junit4]    > Assumption #1: 'monster' test group is disabled (@Monster(value=takes ~20GB-30GB of space and 10 minutes))
   [junit4] Completed [337/421] on J1 in 0.00s, 1 test, 1 skipped
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestPositiveScoresOnlyCollector
   [junit4] Completed [338/421] on J1 in 0.02s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestNewestSegment
   [junit4] Completed [339/421] on J1 in 0.03s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.store.TestByteArrayDataInput
   [junit4] Completed [340/421] on J1 in 0.01s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.Test2BNumericDocValues
   [junit4] IGNOR/A 0.00s J1 | Test2BNumericDocValues.testNumerics
   [junit4]    > Assumption #1: 'monster' test group is disabled (@Monster(value=takes ~ 2 hours if the heap is 5gb))
   [junit4] Completed [341/421] on J1 in 0.00s, 1 test, 1 skipped
   [junit4] 
   [junit4] Suite: org.apache.lucene.analysis.TestReusableStringReader
   [junit4] Completed [343/421] on J1 in 0.01s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.analysis.tokenattributes.TestBytesRefAttImpl
   [junit4] Completed [344/421] on J1 in 0.01s, 1 test
   [junit4] 
   [junit4] HEARTBEAT J0 PID(1142@localhost): 2016-02-21T07:32:57, stalled for  431s at: TestDocumentsWriterStallControl.testAccquireReleaseRace
   [junit4] Suite: org.apache.lucene.codecs.lucene50.TestBlockPostingsFormat
   [junit4] Completed [345/421] on J1 in 20.53s, 32 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.codecs.lucene50.TestBlockPostingsFormat3
   [junit4] Completed [346/421] on J1 in 8.99s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.codecs.lucene50.TestForUtil
   [junit4] Completed [347/421] on J1 in 0.05s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.codecs.lucene50.TestLucene50CompoundFormat
   [junit4] C

[...truncated too long message...]


   [junit4] 
   [junit4] Suite: org.apache.solr.search.RankQueryTest
   [junit4] Completed [520/566 (3!)] on J1 in 1.37s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.solr.search.TestCollapseQParserPlugin
   [junit4] Completed [522/566 (3!)] on J1 in 6.28s, 9 tests
   [junit4] 
   [junit4] Suite: org.apache.solr.search.TestCustomSort
   [junit4] Completed [523/566 (3!)] on J1 in 0.96s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.solr.search.TestFieldSortValues
   [junit4] Completed [524/566 (3!)] on J1 in 0.36s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.solr.search.TestNoOpRegenerator
   [junit4] Completed [525/566 (3!)] on J1 in 0.44s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.solr.search.TestOverriddenPrefixQueryForCustomFieldType
   [junit4] Completed [526/566 (3!)] on J1 in 0.57s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.solr.search.TestSearcherReuse
   [junit4] Completed [527/566 (3!)] on J1 in 0.67s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.solr.search.TestSimpleQParserPlugin
   [junit4] Completed [528/566 (3!)] on J1 in 0.84s, 17 tests
   [junit4] 
   [junit4] Suite: org.apache.solr.search.TestSmileRequest
   [junit4] Completed [529/566 (3!)] on J1 in 4.80s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.solr.search.TestSolr4Spatial2
   [junit4] Completed [530/566 (3!)] on J1 in 0.75s, 4 tests
   [junit4] 
   [junit4] Suite: org.apache.solr.search.TestStandardQParsers
   [junit4] Completed [531/566 (3!)] on J1 in 0.03s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.solr.search.TestTrieFacet
   [junit4] Completed [532/566 (3!)] on J1 in 5.23s, 12 tests
   [junit4] 
   [junit4] Suite: org.apache.solr.search.function.TestSortByMinMaxFunction
   [junit4] Completed [533/566 (3!)] on J1 in 2.37s, 3 tests
   [junit4] 
   [junit4] Suite: org.apache.solr.search.join.BlockJoinFacetDistribTest
   [junit4] Completed [534/566 (3!)] on J1 in 4.92s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.solr.search.join.BlockJoinFacetSimpleTest
   [junit4] Completed [535/566 (3!)] on J1 in 0.65s, 1 test
   [junit4] 
   [junit4] HEARTBEAT J2 PID(3410@lucene1-us-west): 2016-02-21T10:23:12, stalled for  549s at: HdfsBasicDistributedZkTest.test
   [junit4] HEARTBEAT J0 PID(3416@lucene1-us-west): 2016-02-21T10:23:24, stalled for  184s at: TestCloudManagedSchemaConcurrent.test
   [junit4] Suite: org.apache.solr.search.join.TestScoreJoinQPNoScore
   [junit4] Completed [536/566 (3!)] on J1 in 17.64s, 3 tests
   [junit4] 
   [junit4] Suite: org.apache.solr.search.mlt.CloudMLTQParserTest
   [junit4] Completed [537/566 (3!)] on J1 in 38.73s, 2 tests
   [junit4] 
   [junit4] HEARTBEAT J2 PID(3410@lucene1-us-west): 2016-02-21T10:24:12, stalled for  609s at: HdfsBasicDistributedZkTest.test
   [junit4] Suite: org.apache.solr.search.stats.TestDefaultStatsCache
   [junit4] Completed [539/566 (3!)] on J1 in 14.76s, 1 test
   [junit4] 
   [junit4] HEARTBEAT J0 PID(3416@lucene1-us-west): 2016-02-21T10:24:24, stalled for  244s at: TestCloudManagedSchemaConcurrent.test
   [junit4] Suite: org.apache.solr.search.stats.TestExactSharedStatsCache
   [junit4] Completed [540/566 (3!)] on J1 in 12.87s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.solr.security.PKIAuthenticationIntegrationTest
   [junit4] Completed [541/566 (3!)] on J1 in 30.07s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.solr.security.TestPKIAuthenticationPlugin
   [junit4] Completed [542/566 (3!)] on J1 in 0.21s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.solr.spelling.ConjunctionSolrSpellCheckerTest
   [junit4] Completed [543/566 (3!)] on J1 in 0.02s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.solr.spelling.suggest.TestAnalyzeInfixSuggestions
   [junit4] Completed [544/566 (3!)] on J1 in 2.47s, 4 tests
   [junit4] 
   [junit4] Suite: org.apache.solr.spelling.suggest.TestBlendedInfixSuggestions
   [junit4] Completed [545/566 (3!)] on J1 in 0.81s, 6 tests
   [junit4] 
   [junit4] Suite: org.apache.solr.spelling.suggest.TestHighFrequencyDictionaryFactory
   [junit4] Completed [546/566 (3!)] on J1 in 0.60s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.solr.store.blockcache.BlockCacheTest
   [junit4] Completed [547/566 (3!)] on J1 in 0.41s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.solr.schema.TestCloudManagedSchemaConcurrent
   [junit4] Completed [548/566 (3!)] on J0 in 289.39s, 1 test
   [junit4] 
   [junit4] HEARTBEAT J2 PID(3410@lucene1-us-west): 2016-02-21T10:25:12, stalled for  669s at: HdfsBasicDistributedZkTest.test
   [junit4] Suite: org.apache.solr.update.TestDocBasedVersionConstraints
   [junit4] Completed [549/566 (3!)] on J1 in 12.54s, 6 tests
   [junit4] 
   [junit4] Suite: org.apache.solr.update.VersionInfoTest
   [junit4] Completed [550/566 (3!)] on J1 in 0.96s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.solr.update.processor.CloneFieldUpdateProcessorFactoryTest
   [junit4] Completed [551/566 (3!)] on J1 in 1.11s, 9 tests
   [junit4] 
   [junit4] Suite: org.apache.solr.update.TestHdfsUpdateLog
   [junit4] Completed [552/566 (3!)] on J0 in 21.40s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.solr.update.processor.DocExpirationUpdateProcessorFactoryTest
   [junit4] Completed [553/566 (3!)] on J1 in 9.72s, 5 tests
   [junit4] 
   [junit4] Suite: org.apache.solr.update.processor.IgnoreCommitOptimizeUpdateProcessorFactoryTest
   [junit4] Completed [554/566 (3!)] on J0 in 1.07s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.solr.update.processor.PreAnalyzedUpdateProcessorTest
   [junit4] Completed [555/566 (3!)] on J1 in 0.89s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.solr.update.processor.UUIDUpdateProcessorFallbackTest
   [junit4] Completed [556/566 (3!)] on J1 in 0.87s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.solr.util.TestRTimerTree
   [junit4] Completed [557/566 (3!)] on J1 in 0.02s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.solr.util.TestSolrCLIRunExample
   [junit4] Completed [558/566 (3!)] on J1 in 20.45s, 3 tests, 1 skipped
   [junit4] 
   [junit4] Suite: org.apache.solr.util.hll.BigEndianAscendingWordDeserializerTest
   [junit4] Completed [559/566 (3!)] on J1 in 1.44s, 4 tests
   [junit4] 
   [junit4] Suite: org.apache.solr.util.hll.BigEndianAscendingWordSerializerTest
   [junit4] Completed [560/566 (3!)] on J1 in 0.04s, 5 tests
   [junit4] 
   [junit4] Suite: org.apache.solr.util.hll.BitVectorTest
   [junit4] Completed [561/566 (3!)] on J1 in 0.04s, 4 tests
   [junit4] 
   [junit4] Suite: org.apache.solr.util.hll.HLLSerializationTest
   [junit4] IGNOR/A 0.00s J1 | HLLSerializationTest.manyValuesHLLSerializationTest
   [junit4]    > Assumption #1: 'monster' test group is disabled (@Monster(value=may require as much as -Dtests.heapsize=4g depending on random values picked))
   [junit4] IGNOR/A 0.00s J1 | HLLSerializationTest.monsterHLLSerializationTest
   [junit4]    > Assumption #1: 'monster' test group is disabled (@Monster(value=needs roughly -Dtests.heapsize=8g because of the (multiple) massive data structs))
   [junit4] IGNOR/A 0.00s J1 | HLLSerializationTest.manyValuesMonsterHLLSerializationTest
   [junit4]    > Assumption #1: 'monster' test group is disabled (@Monster(value=can require as much as -Dtests.heapsize=4g because of the massive data structs))
   [junit4] Completed [562/566 (3!)] on J1 in 10.90s, 4 tests, 3 skipped
   [junit4] 
   [junit4] Suite: org.apache.solr.util.hll.HLLUtilTest
   [junit4] Completed [563/566 (3!)] on J1 in 0.02s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.solr.util.hll.SparseHLLTest
   [junit4] Completed [564/566 (3!)] on J1 in 0.11s, 8 tests
   [junit4] 
   [junit4] Suite: org.apache.solr.update.processor.TestNamedUpdateProcessors
   [junit4] Completed [565/566 (3!)] on J0 in 36.81s, 1 test
   [junit4] 
   [junit4] HEARTBEAT J2 PID(3410@lucene1-us-west): 2016-02-21T10:26:12, stalled for  729s at: HdfsBasicDistributedZkTest.test
   [junit4] Suite: org.apache.solr.cloud.hdfs.HdfsBasicDistributedZkTest
   [junit4] Completed [566/566 (3!)] on J2 in 768.87s, 1 test
   [junit4] 
   [junit4] 
   [junit4] Tests with failures [seed: F181951231137BA6]:
   [junit4]   - org.apache.solr.cloud.BasicDistributedZk2Test.test
   [junit4]   - org.apache.solr.handler.TestReplicationHandler.doTestIndexAndConfigAliasReplication
   [junit4]   - org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test
   [junit4] 
   [junit4] 
   [junit4] JVM J0:     0.62 ..  4233.96 =  4233.34s
   [junit4] JVM J1:     0.63 ..  4232.35 =  4231.71s
   [junit4] JVM J2:     0.64 ..  4274.76 =  4274.12s
   [junit4] Execution time total: 1 hour 11 minutes 15 seconds
   [junit4] Tests summary: 566 suites (7 ignored), 2279 tests, 3 errors, 34 ignored (18 assumptions)

BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:757: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:694: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:534: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1477: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1033: There were test failures: 566 suites (7 ignored), 2279 tests, 3 errors, 34 ignored (18 assumptions) [seed: F181951231137BA6]

Total time: 208 minutes 39 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1102 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1102/

1 tests failed.
FAILED:  org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=58380, name=collection4, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=58380, name=collection4, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:37521: collection already exists: awholynewstresscollection_collection4_1
	at __randomizedtesting.SeedInfo.seed([A56B512213E73DFE]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:576)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:240)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:229)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1573)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1594)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:960)




Build Log:
[...truncated 11875 lines...]
   [junit4] JVM J2: stdout was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J2-20160220_095809_482.sysout
   [junit4] >>> JVM J2 emitted unexpected output (verbatim) ----
   [junit4] java.lang.OutOfMemoryError: GC overhead limit exceeded
   [junit4] Dumping heap to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/heapdumps/java_pid20839.hprof ...
   [junit4] Heap dump file created [642206365 bytes in 14.577 secs]
   [junit4] <<< JVM J2: EOF ----

   [junit4] JVM J2: stderr was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J2-20160220_095809_482.syserr
   [junit4] >>> JVM J2 emitted unexpected output (verbatim) ----
   [junit4] WARN: Unhandled exception in event serialization. -> java.lang.OutOfMemoryError: GC overhead limit exceeded
   [junit4] <<< JVM J2: EOF ----

   [junit4] Suite: org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_A56B512213E73DFE-001/init-core-data-001
   [junit4]   2> 1521429 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[A56B512213E73DFE]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false)
   [junit4]   2> 1521429 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[A56B512213E73DFE]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   1> Formatting using clusterid: testClusterID
   [junit4]   2> 1521467 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[A56B512213E73DFE]-worker) [    ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
   [junit4]   2> 1521475 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[A56B512213E73DFE]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 1521479 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[A56B512213E73DFE]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 1521496 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[A56B512213E73DFE]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/hdfs to ./temp/Jetty_localhost_44696_hdfs____lc483h/webapp
   [junit4]   2> 1521615 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[A56B512213E73DFE]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 1521883 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[A56B512213E73DFE]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:44696
   [junit4]   2> 1521950 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[A56B512213E73DFE]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 1521952 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[A56B512213E73DFE]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 1521967 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[A56B512213E73DFE]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_45050_datanode____5d18x/webapp
   [junit4]   2> 1522084 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[A56B512213E73DFE]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 1522336 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[A56B512213E73DFE]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:45050
   [junit4]   2> 1522377 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[A56B512213E73DFE]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 1522379 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[A56B512213E73DFE]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 1522391 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[A56B512213E73DFE]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_57792_datanode____.51ib8r/webapp
   [junit4]   2> 1522436 INFO  (IPC Server handler 3 on 49856) [    ] BlockStateChange BLOCK* processReport: from storage DS-fe98c040-7775-4a60-beea-faec88af38d3 node DatanodeRegistration(127.0.0.1, datanodeUuid=be3b4da8-efd7-4136-b3d1-fde90eb20a8b, infoPort=45050, ipcPort=53542, storageInfo=lv=-56;cid=testClusterID;nsid=367502049;c=0), blocks: 0, hasStaleStorages: true, processing time: 0 msecs
   [junit4]   2> 1522436 INFO  (IPC Server handler 3 on 49856) [    ] BlockStateChange BLOCK* processReport: from storage DS-6b7df692-a675-410f-9f85-41916c115914 node DatanodeRegistration(127.0.0.1, datanodeUuid=be3b4da8-efd7-4136-b3d1-fde90eb20a8b, infoPort=45050, ipcPort=53542, storageInfo=lv=-56;cid=testClusterID;nsid=367502049;c=0), blocks: 0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 1522520 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[A56B512213E73DFE]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 1522781 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[A56B512213E73DFE]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:57792
   [junit4]   2> 1522866 INFO  (IPC Server handler 8 on 49856) [    ] BlockStateChange BLOCK* processReport: from storage DS-1f3d6b97-e935-4442-9ab7-b2913d481de9 node DatanodeRegistration(127.0.0.1, datanodeUuid=a2e0ebf4-4ebd-4a1e-9e79-6cdf2e29340e, infoPort=57792, ipcPort=60302, storageInfo=lv=-56;cid=testClusterID;nsid=367502049;c=0), blocks: 0, hasStaleStorages: true, processing time: 0 msecs
   [junit4]   2> 1522866 INFO  (IPC Server handler 8 on 49856) [    ] BlockStateChange BLOCK* processReport: from storage DS-29b92ce8-aeab-4bbe-bf18-d9d55ab34c04 node DatanodeRegistration(127.0.0.1, datanodeUuid=a2e0ebf4-4ebd-4a1e-9e79-6cdf2e29340e, infoPort=57792, ipcPort=60302, storageInfo=lv=-56;cid=testClusterID;nsid=367502049;c=0), blocks: 0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 1522947 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 1522947 INFO  (Thread-51233) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1522947 INFO  (Thread-51233) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 1523047 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [    ] o.a.s.c.ZkTestServer start zk server on port:52913
   [junit4]   2> 1523047 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1523048 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1523051 INFO  (zkCallback-1353-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@167b42f1 name:ZooKeeperConnection Watcher:127.0.0.1:52913 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1523051 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1523051 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1523051 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 1523054 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1523054 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1523056 INFO  (zkCallback-1354-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7176e7ff name:ZooKeeperConnection Watcher:127.0.0.1:52913/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1523056 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1523056 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1523056 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 1523058 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 1523060 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 1523062 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 1523063 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 1523063 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 1523066 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 1523066 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 1523068 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1523068 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1523070 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 1523070 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 1523071 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 1523071 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 1523073 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 1523073 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 1523075 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 1523075 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 1523076 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1523076 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1523078 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1523078 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1523079 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 1523080 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 1523081 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 1523081 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 1523170 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_A56B512213E73DFE-001/control-001/cores/collection1
   [junit4]   2> 1523171 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 1523194 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@3f9347a5{/,null,AVAILABLE}
   [junit4]   2> 1523195 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [    ] o.e.j.s.ServerConnector Started ServerConnector@5e44c23e{HTTP/1.1}{127.0.0.1:35990}
   [junit4]   2> 1523195 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [    ] o.e.j.s.Server Started @1526221ms
   [junit4]   2> 1523195 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/, hostPort=35990, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_A56B512213E73DFE-001/control-001/cores, solr.data.dir=hdfs://localhost:49856/hdfs__localhost_49856__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-5.x_solr_build_solr-core_test_J1_temp_solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_A56B512213E73DFE-001_tempDir-002_control_data}
   [junit4]   2> 1523195 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@7b3cb2c6
   [junit4]   2> 1523195 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_A56B512213E73DFE-001/control-001'
   [junit4]   2> 1523196 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1523196 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1523196 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1523207 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1523213 INFO  (zkCallback-1355-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@760b5698 name:ZooKeeperConnection Watcher:127.0.0.1:52913/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1523213 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1523213 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1523217 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1523217 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_A56B512213E73DFE-001/control-001/solr.xml
   [junit4]   2> 1523232 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_A56B512213E73DFE-001/control-001/cores
   [junit4]   2> 1523232 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [    ] o.a.s.c.CoreContainer New CoreContainer 956287934
   [junit4]   2> 1523232 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_A56B512213E73DFE-001/control-001]
   [junit4]   2> 1523232 WARN  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_A56B512213E73DFE-001/control-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_A56B512213E73DFE-001/control-001/lib
   [junit4]   2> 1523233 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1523234 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1523234 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1523234 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1523234 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:52913/solr
   [junit4]   2> 1523234 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1523235 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1523249 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1523257 INFO  (zkCallback-1358-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@430e9aa1 name:ZooKeeperConnection Watcher:127.0.0.1:52913 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1523258 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1523258 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1523263 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [n:127.0.0.1:35990_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1523265 INFO  (zkCallback-1359-thread-1-processing-n:127.0.0.1:35990_) [n:127.0.0.1:35990_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7369b97a name:ZooKeeperConnection Watcher:127.0.0.1:52913/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1523266 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [n:127.0.0.1:35990_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1523267 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [n:127.0.0.1:35990_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 1523268 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [n:127.0.0.1:35990_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 1523272 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [n:127.0.0.1:35990_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 1523274 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [n:127.0.0.1:35990_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 1523276 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [n:127.0.0.1:35990_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 1523280 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [n:127.0.0.1:35990_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 1523282 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [n:127.0.0.1:35990_    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 1523283 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [n:127.0.0.1:35990_    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 1523285 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [n:127.0.0.1:35990_    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 1523286 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [n:127.0.0.1:35990_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:35990_
   [junit4]   2> 1523286 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [n:127.0.0.1:35990_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:35990_
   [junit4]   2> 1523288 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [n:127.0.0.1:35990_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 1523290 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [n:127.0.0.1:35990_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 1523291 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [n:127.0.0.1:35990_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1523292 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [n:127.0.0.1:35990_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/95418044618637316-127.0.0.1:35990_-n_0000000000
   [junit4]   2> 1523292 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [n:127.0.0.1:35990_    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:35990_
   [junit4]   2> 1523292 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [n:127.0.0.1:35990_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 1523294 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [n:127.0.0.1:35990_    ] o.a.s.c.Overseer Overseer (id=95418044618637316-127.0.0.1:35990_-n_0000000000) starting
   [junit4]   2> 1523296 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [n:127.0.0.1:35990_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 1523302 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [n:127.0.0.1:35990_    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 1523303 INFO  (OverseerCollectionConfigSetProcessor-95418044618637316-127.0.0.1:35990_-n_0000000000) [n:127.0.0.1:35990_    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 1523303 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [n:127.0.0.1:35990_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1523304 INFO  (OverseerStateUpdate-95418044618637316-127.0.0.1:35990_-n_0000000000) [n:127.0.0.1:35990_    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 1523318 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [n:127.0.0.1:35990_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1523318 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [n:127.0.0.1:35990_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1523319 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [n:127.0.0.1:35990_    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_A56B512213E73DFE-001/control-001/cores
   [junit4]   2> 1523320 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [n:127.0.0.1:35990_    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {coreNodeName=, config=solrconfig.xml, loadOnStartup=true, name=collection1, transient=false, configSetProperties=configsetprops.json, shard=, dataDir=data/, collection=control_collection, schema=schema.xml}
   [junit4]   2> 1523320 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [n:127.0.0.1:35990_    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_A56B512213E73DFE-001/control-001/cores/collection1
   [junit4]   2> 1523320 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [n:127.0.0.1:35990_    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 1523321 INFO  (coreLoadExecutor-3709-thread-1-processing-n:127.0.0.1:35990_) [n:127.0.0.1:35990_ c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 1523322 INFO  (coreLoadExecutor-3709-thread-1-processing-n:127.0.0.1:35990_) [n:127.0.0.1:35990_ c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1523322 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [n:127.0.0.1:35990_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 1523322 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [n:127.0.0.1:35990_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1523324 INFO  (OverseerStateUpdate-95418044618637316-127.0.0.1:35990_-n_0000000000) [n:127.0.0.1:35990_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "shard":null,
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:35990",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "node_name":"127.0.0.1:35990_",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 1523324 INFO  (coreLoadExecutor-3709-thread-1-processing-n:127.0.0.1:35990_) [n:127.0.0.1:35990_ c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 1523324 INFO  (OverseerStateUpdate-95418044618637316-127.0.0.1:35990_-n_0000000000) [n:127.0.0.1:35990_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "shard":null,
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:35990",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "node_name":"127.0.0.1:35990_",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 1523325 INFO  (OverseerStateUpdate-95418044618637316-127.0.0.1:35990_-n_0000000000) [n:127.0.0.1:35990_    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 1523325 INFO  (OverseerStateUpdate-95418044618637316-127.0.0.1:35990_-n_0000000000) [n:127.0.0.1:35990_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 1523327 INFO  (zkCallback-1359-thread-1-processing-n:127.0.0.1:35990_) [n:127.0.0.1:35990_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1524325 INFO  (coreLoadExecutor-3709-thread-1-processing-n:127.0.0.1:35990_) [n:127.0.0.1:35990_ c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1524325 INFO  (coreLoadExecutor-3709-thread-1-processing-n:127.0.0.1:35990_) [n:127.0.0.1:35990_ c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 1524325 INFO  (coreLoadExecutor-3709-thread-1-processing-n:127.0.0.1:35990_) [n:127.0.0.1:35990_ c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 1524325 INFO  (coreLoadExecutor-3709-thread-1-processing-n:127.0.0.1:35990_) [n:127.0.0.1:35990_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/control_collection]
   [junit4]   2> 1524326 INFO  (coreLoadExecutor-3709-thread-1-processing-n:127.0.0.1:35990_) [n:127.0.0.1:35990_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/control_collection] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 1524326 INFO  (coreLoadExecutor-3709-thread-1-processing-n:127.0.0.1:35990_) [n:127.0.0.1:35990_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_A56B512213E73DFE-001/control-001/cores/collection1'
   [junit4]   2> 1524326 INFO  (coreLoadExecutor-3709-thread-1-processing-n:127.0.0.1:35990_) [n:127.0.0.1:35990_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1524326 INFO  (coreLoadExecutor-3709-thread-1-processing-n:127.0.0.1:35990_) [n:127.0.0.1:35990_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1524328 INFO  (coreLoadExecutor-3709-thread-1-processing-n:127.0.0.1:35990_) [n:127.0.0.1:35990_ c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 1524334 INFO  (coreLoadExecutor-3709-thread-1-processing-n:127.0.0.1:35990_) [n:127.0.0.1:35990_ c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 1524347 INFO  (coreLoadExecutor-3709-thread-1-processing-n:127.0.0.1:35990_) [n:127.0.0.1:35990_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.5.0
   [junit4]   2> 1524376 INFO  (coreLoadExecutor-3709-thread-1-processing-n:127.0.0.1:35990_) [n:127.0.0.1:35990_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1524383 INFO  (coreLoadExecutor-3709-thread-1-processing-n:127.0.0.1:35990_) [n:127.0.0.1:35990_ c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1524640 INFO  (coreLoadExecutor-3709-thread-1-processing-n:127.0.0.1:35990_) [n:127.0.0.1:35990_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 1524648 INFO  (coreLoadExecutor-3709-thread-1-processing-n:127.0.0.1:35990_) [n:127.0.0.1:35990_ c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 1524650 INFO  (coreLoadExecutor-3709-thread-1-processing-n:127.0.0.1:35990_) [n:127.0.0.1:35990_ c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 1524659 INFO  (coreLoadExecutor-3709-thread-1-processing-n:127.0.0.1:35990_) [n:127.0.0.1:35990_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1524661 INFO  (coreLoadExecutor-3709-thread-1-processing-n:127.0.0.1:35990_) [n:127.0.0.1:35990_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1524664 INFO  (coreLoadExecutor-3709-thread-1-processing-n:127.0.0.1:35990_) [n:127.0.0.1:35990_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1524665 INFO  (coreLoadExecutor-3709-thread-1-processing-n:127.0.0.1:35990_) [n:127.0.0.1:35990_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1524666 INFO  (coreLoadExecutor-3709-thread-1-processing-n:127.0.0.1:35990_) [n:127.0.0.1:35990_ c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 1524667 INFO  (coreLoadExecutor-3709-thread-1-processing-n:127.0.0.1:35990_) [n:127.0.0.1:35990_ c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 1524667 INFO  (coreLoadExecutor-3709-thread-1-processing-n:127.0.0.1:35990_) [n:127.0.0.1:35990_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.HdfsDirectoryFactory
   [junit4]   2> 1524667 INFO  (coreLoadExecutor-3709-thread-1-processing-n:127.0.0.1:35990_) [n:127.0.0.1:35990_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:49856/solr_hdfs_home
   [junit4]   2> 1524667 INFO  (coreLoadExecutor-3709-thread-1-processing-n:127.0.0.1:35990_) [n:127.0.0.1:35990_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
   [junit4]   2> 1524667 INFO  (coreLoadExecutor-3709-thread-1-processing-n:127.0.0.1:35990_) [n:127.0.0.1:35990_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_A56B512213E73DFE-001/control-001/cores/collection1], dataDir=[hdfs://localhost:49856/solr_hdfs_home/control_collection/core_node1/data/]
   [junit4]   2> 1524667 INFO  (coreLoadExecutor-3709-thread-1-processing-n:127.0.0.1:35990_) [n:127.0.0.1:35990_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@39e3326
   [junit4]   2> 1524667 INFO  (coreLoadExecutor-3709-thread-1-processing-n:127.0.0.1:35990_) [n:127.0.0.1:35990_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:49856/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 1524682 INFO  (coreLoadExecutor-3709-thread-1-processing-n:127.0.0.1:35990_) [n:127.0.0.1:35990_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:49856/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 1524683 INFO  (coreLoadExecutor-3709-thread-1-processing-n:127.0.0.1:35990_) [n:127.0.0.1:35990_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=hdfs://localhost:49856/solr_hdfs_home/control_collection/core_node1/data/index/
   [junit4]   2> 1524693 WARN  (coreLoadExecutor-3709-thread-1-processing-n:127.0.0.1:35990_) [n:127.0.0.1:35990_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory 'hdfs:/localhost:49856/solr_hdfs_home/control_collection/core_node1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 1524693 INFO  (coreLoadExecutor-3709-thread-1-processing-n:127.0.0.1:35990_) [n:127.0.0.1:35990_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:49856/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 1524704 INFO  (coreLoadExecutor-3709-thread-1-processing-n:127.0.0.1:35990_) [n:127.0.0.1:35990_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:49856/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 1524704 INFO  (coreLoadExecutor-3709-thread-1-processing-n:127.0.0.1:35990_) [n:127.0.0.1:35990_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@2621ff6d
   [junit4]   2> 1524718 INFO  (IPC Server handler 9 on 49856) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:45519 is added to blk_1073741825_1001{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-29b92ce8-aeab-4bbe-bf18-d9d55ab34c04:NORMAL:127.0.0.1:57350|RBW], ReplicaUnderConstruction[[DISK]DS-6b7df692-a675-410f-9f85-41916c115914:NORMAL:127.0.0.1:45519|FINALIZED]]} size 0
   [junit4]   2> 1524719 INFO  (IPC Server handler 0 on 49856) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:57350 is added to blk_1073741825_1001{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-29b92ce8-aeab-4bbe-bf18-d9d55ab34c04:NORMAL:127.0.0.1:57350|RBW], ReplicaUnderConstruction[[DISK]DS-6b7df692-a675-410f-9f85-41916c115914:NORMAL:127.0.0.1:45519|FINALIZED]]} size 0
   [junit4]   2> 1524721 INFO  (coreLoadExecutor-3709-thread-1-processing-n:127.0.0.1:35990_) [n:127.0.0.1:35990_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(HdfsDirectory@hdfs://localhost:49856/solr_hdfs_home/control_collection/core_node1/data/index lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@351b5b52; maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 1524721 INFO  (coreLoadExecutor-3709-thread-1-processing-n:127.0.0.1:35990_) [n:127.0.0.1:35990_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 1524722 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:35990_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in hdfs://localhost:49856/solr_hdfs_home/control_collection/core_node1/data/
   [junit4]   2> 1524726 INFO  (coreLoadExecutor-3709-thread-1-processing-n:127.0.0.1:35990_) [n:127.0.0.1:35990_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 1524726 INFO  (coreLoadExecutor-3709-thread-1-processing-n:127.0.0.1:35990_) [n:127.0.0.1:35990_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 1524726 INFO  (coreLoadExecutor-3709-thread-1-processing-n:127.0.0.1:35990_) [n:127.0.0.1:35990_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 1524727 INFO  (coreLoadExecutor-3709-thread-1-processing-n:127.0.0.1:35990_) [n:127.0.0.1:35990_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1524727 INFO  (coreLoadExecutor-3709-thread-1-processing-n:127.0.0.1:35990_) [n:127.0.0.1:35990_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1524727 INFO  (coreLoadExecutor-3709-thread-1-processing-n:127.0.0.1:35990_) [n:127.0.0.1:35990_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 1524727 INFO  (coreLoadExecutor-3709-thread-1-processing-n:127.0.0.1:35990_) [n:127.0.0.1:35990_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1524727 INFO  (coreLoadExecutor-3709-thread-1-processing-n:127.0.0.1:35990_) [n:127.0.0.1:35990_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1524727 INFO  (coreLoadExecutor-3709-thread-1-processing-n:127.0.0.1:35990_) [n:127.0.0.1:35990_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 1524727 WARN  (coreLoadExecutor-3709-thread-1-processing-n:127.0.0.1:35990_) [n:127.0.0.1:35990_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
   [junit4]   2> 1524730 INFO  (coreLoadExecutor-3709-thread-1-processing-n:127.0.0.1:35990_) [n:127.0.0.1:35990_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1524731 INFO  (coreLoadExecutor-3709-thread-1-processing-n:127.0.0.1:35990_) [n:127.0.0.1:35990_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1524733 INFO  (coreLoadExecutor-3709-thread-1-processing-n:127.0.0.1:35990_) [n:127.0.0.1:35990_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1524734 INFO  (coreLoadExecutor-3709-thread-1-processing-n:127.0.0.1:35990_) [n:127.0.0.1:35990_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1524738 INFO  (coreLoadExecutor-3709-thread-1-processing-n:127.0.0.1:35990_) [n:127.0.0.1:35990_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/file,/admin/plugins,/get,/admin/ping,/update/json/docs,/replication,/config,/admin/properties,/admin/mbeans,/update/json,/admin/luke,/dump,/admin/logging,standard,/update,/schema,/update/csv,/admin/system,/admin/threads,/admin/segments
   [junit4]   2> 1524739 INFO  (coreLoadExecutor-3709-thread-1-processing-n:127.0.0.1:35990_) [n:127.0.0.1:35990_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 1524740 INFO  (coreLoadExecutor-3709-thread-1-processing-n:127.0.0.1:35990_) [n:127.0.0.1:35990_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
   [junit4]   2> 1524740 INFO  (coreLoadExecutor-3709-thread-1-processing-n:127.0.0.1:35990_) [n:127.0.0.1:35990_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 tlogDfsReplication=2
   [junit4]   2> 1524757 INFO  (coreLoadExecutor-3709-thread-1-processing-n:127.0.0.1:35990_) [n:127.0.0.1:35990_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1524757 INFO  (coreLoadExecutor-3709-thread-1-processing-n:127.0.0.1:35990_) [n:127.0.0.1:35990_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1524758 INFO  (coreLoadExecutor-3709-thread-1-processing-n:127.0.0.1:35990_) [n:127.0.0.1:35990_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@49614f66
   [junit4]   2> 1524763 INFO  (coreLoadExecutor-3709-thread-1-processing-n:127.0.0.1:35990_) [n:127.0.0.1:35990_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(HdfsDirectory@hdfs://localhost:49856/solr_hdfs_home/control_collection/core_node1/data/index lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@351b5b52; maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 1524763 INFO  (coreLoadExecutor-3709-thread-1-processing-n:127.0.0.1:35990_) [n:127.0.0.1:35990_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 1524763 INFO  (coreLoadExecutor-3709-thread-1-processing-n:127.0.0.1:35990_) [n:127.0.0.1:35990_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@33da25de[collection1] main
   [junit4]   2> 1524763 INFO  (coreLoadExecutor-3709-thread-1-processing-n:127.0.0.1:35990_) [n:127.0.0.1:35990_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/control_collection]
   [junit4]   2> 1524764 INFO  (coreLoadExecutor-3709-thread-1-processing-n:127.0.0.1:35990_) [n:127.0.0.1:35990_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/control_collection] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 1524764 INFO  (coreLoadExecutor-3709-thread-1-processing-n:127.0.0.1:35990_) [n:127.0.0.1:35990_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 1524764 INFO  (coreLoadExecutor-3709-thread-1-processing-n:127.0.0.1:35990_) [n:127.0.0.1:35990_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1524764 INFO  (coreLoadExecutor-3709-thread-1-processing-n:127.0.0.1:35990_) [n:127.0.0.1:35990_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 1524764 INFO  (coreLoadExecutor-3709-thread-1-processing-n:127.0.0.1:35990_) [n:127.0.0.1:35990_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1524765 INFO  (coreLoadExecutor-3709-thread-1-processing-n:127.0.0.1:35990_) [n:127.0.0.1:35990_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 1524765 INFO  (coreLoadExecutor-3709-thread-1-processing-n:127.0.0.1:35990_) [n:127.0.0.1:35990_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1524765 INFO  (coreLoadExecutor-3709-thread-1-processing-n:127.0.0.1:35990_) [n:127.0.0.1:35990_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 1524765 INFO  (coreLoadExecutor-3709-thread-1-processing-n:127.0.0.1:35990_) [n:127.0.0.1:35990_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1524766 INFO  (searcherExecutor-3710-thread-1-processing-r:core_node1 s:shard1 n:127.0.0.1:35990_ c:control_collection x:collection1) [n:127.0.0.1:35990_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@33da25de[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1524768 WARN  (coreLoadExecutor-3709-thread-1-processing-n:127.0.0.1:35990_) [n:127.0.0.1:35990_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.h.HdfsLocalityReporter Could not retrieve locality information for hdfs://localhost:37249/solr3 due to exception: java.net.ConnectException: Call From lucene1-us-west/10.41.0.5 to localhost:37249 failed on connection exception: java.net.ConnectException: Connection refused; For more details see:  http://wiki.apache.org/hadoop/ConnectionRefused
   [junit4]   2> 1524769 INFO  (coreLoadExecutor-3709-thread-1-processing-n:127.0.0.1:35990_) [n:127.0.0.1:35990_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 1524769 INFO  (coreLoadExecutor-3709-thread-1-processing-n:127.0.0.1:35990_) [n:127.0.0.1:35990_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 256 version buckets from index
   [junit4]   2> 1524769 INFO  (coreLoadExecutor-3709-thread-1-processing-n:127.0.0.1:35990_) [n:127.0.0.1:35990_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 1524769 INFO  (coreLoadExecutor-3709-thread-1-processing-n:127.0.0.1:35990_) [n:127.0.0.1:35990_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1526688715807653888
   [junit4]   2> 1524769 INFO  (coreLoadExecutor-3709-thread-1-processing-n:127.0.0.1:35990_) [n:127.0.0.1:35990_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 0.0ms to seed version buckets with highest version 1526688715807653888
   [junit4]   2> 1524769 INFO  (coreLoadExecutor-3709-thread-1-processing-n:127.0.0.1:35990_) [n:127.0.0.1:35990_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 1524770 INFO  (coreLoadExecutor-3709-thread-1-processing-n:127.0.0.1:35990_) [n:127.0.0.1:35990_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 1524770 INFO  (coreZkRegister-3703-thread-1-processing-r:core_node1 s:shard1 n:127.0.0.1:35990_ c:control_collection x:collection1) [n:127.0.0.1:35990_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:35990 collection:control_collection shard:shard1
   [junit4]   2> 1524771 INFO  (coreZkRegister-3703-thread-1-processing-r:core_node1 s:shard1 n:127.0.0.1:35990_ c:control_collection x:collection1) [n:127.0.0.1:35990_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 1524774 INFO  (coreZkRegister-3703-thread-1-processing-r:core_node1 s:shard1 n:127.0.0.1:35990_ c:control_collection x:collection1) [n:127.0.0.1:35990_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/95418044618637316-core_node1-n_0000000000
   [junit4]   2> 1524775 INFO  (coreZkRegister-3703-thread-1-processing-r:core_node1 s:shard1 n:127.0.0.1:35990_ c:control_collection x:collection1) [n:127.0.0.1:35990_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard=shard1 and weAreReplacement=false and leaderVoteWait=180000
   [junit4]   2> 1524777 INFO  (coreZkRegister-3703-thread-1-processing-r:core_node1 s:shard1 n:127.0.0.1:35990_ c:control_collection x:collection1) [n:127.0.0.1:35990_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1524777 INFO  (coreZkRegister-3703-thread-1-processing-r:core_node1 s:shard1 n:127.0.0.1:35990_ c:control_collection x:collection1) [n:127.0.0.1:35990_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1524777 INFO  (OverseerStateUpdate-95418044618637316-127.0.0.1:35990_-n_0000000000) [n:127.0.0.1:35990_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 1524777 INFO  (coreZkRegister-3703-thread-1-processing-r:core_node1 s:shard1 n:127.0.0.1:35990_ c:control_collection x:collection1) [n:127.0.0.1:35990_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:35990/collection1/
   [junit4]   2> 1524777 INFO  (coreZkRegister-3703-thread-1-processing-r:core_node1 s:shard1 n:127.0.0.1:35990_ c:control_collection x:collection1) [n:127.0.0.1:35990_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 1524777 INFO  (coreZkRegister-3703-thread-1-processing-r:core_node1 s:shard1 n:127.0.0.1:35990_ c:control_collection x:collection1) [n:127.0.0.1:35990_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:35990/collection1/ has no replicas
   [junit4]   2> 1524778 INFO  (coreZkRegister-3703-thread-1-processing-r:core_node1 s:shard1 n:127.0.0.1:35990_ c:control_collection x:collection1) [n:127.0.0.1:35990_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 1524781 INFO  (coreZkRegister-3703-thread-1-processing-r:core_node1 s:shard1 n:127.0.0.1:35990_ c:control_collection x:collection1) [n:127.0.0.1:35990_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 1524783 INFO  (coreZkRegister-3703-thread-1-processing-r:core_node1 s:shard1 n:127.0.0.1:35990_ c:control_collection x:collection1) [n:127.0.0.1:35990_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:35990/collection1/ shard1
   [junit4]   2> 1524783 INFO  (OverseerStateUpdate-95418044618637316-127.0.0.1:35990_-n_0000000000) [n:127.0.0.1:35990_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:35990",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 1
   [junit4]   2> 1524827 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1524827 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1524829 INFO  (zkCallback-1361-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4bdd7336 name:ZooKeeperConnection Watcher:127.0.0.1:52913/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1524829 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1524829 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1524829 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1524831 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 1524885 INFO  (zkCallback-1359-thread-1-processing-n:127.0.0.1:35990_) [n:127.0.0.1:35990_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1524886 INFO  (zkCallback-1361-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1524920 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_A56B512213E73DFE-001/shard-1-001/cores/collection1
   [junit4]   2> 1524920 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_A56B512213E73DFE-001/shard-1-001
   [junit4]   2> 1524921 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 1524923 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@385415b9{/,null,AVAILABLE}
   [junit4]   2> 1524924 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [    ] o.e.j.s.ServerConnector Started ServerConnector@54232664{HTTP/1.1}{127.0.0.1:51863}
   [junit4]   2> 1524924 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [    ] o.e.j.s.Server Started @1527950ms
   [junit4]   2> 1524924 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/, solr.data.dir=hdfs://localhost:49856/hdfs__localhost_49856__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-5.x_solr_build_solr-core_test_J1_temp_solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_A56B512213E73DFE-001_tempDir-002_jetty1, hostPort=51863, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_A56B512213E73DFE-001/shard-1-001/cores, solrconfig=solrconfig.xml}
   [junit4]   2> 1524924 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@7b3cb2c6
   [junit4]   2> 1524924 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_A56B512213E73DFE-001/shard-1-001'
   [junit4]   2> 1524925 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1524925 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1524925 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1524925 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1524927 INFO  (zkCallback-1362-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1df817ac name:ZooKeeperConnection Watcher:127.0.0.1:52913/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1524927 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1524927 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1524928 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1524928 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_A56B512213E73DFE-001/shard-1-001/solr.xml
   [junit4]   2> 1524934 INFO  (coreZkRegister-3703-thread-1-processing-r:core_node1 s:shard1 n:127.0.0.1:35990_ c:control_collection x:collection1) [n:127.0.0.1:35990_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:35990/collection1/ and leader is http://127.0.0.1:35990/collection1/
   [junit4]   2> 1524934 INFO  (coreZkRegister-3703-thread-1-processing-r:core_node1 s:shard1 n:127.0.0.1:35990_ c:control_collection x:collection1) [n:127.0.0.1:35990_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:35990
   [junit4]   2> 1524934 INFO  (coreZkRegister-3703-thread-1-processing-r:core_node1 s:shard1 n:127.0.0.1:35990_ c:control_collection x:collection1) [n:127.0.0.1:35990_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1524934 INFO  (coreZkRegister-3703-thread-1-processing-r:core_node1 s:shard1 n:127.0.0.1:35990_ c:control_collection x:collection1) [n:127.0.0.1:35990_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 1524934 INFO  (coreZkRegister-3703-thread-1-processing-r:core_node1 s:shard1 n:127.0.0.1:35990_ c:control_collection x:collection1) [n:127.0.0.1:35990_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1524936 INFO  (OverseerStateUpdate-95418044618637316-127.0.0.1:35990_-n_0000000000) [n:127.0.0.1:35990_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:35990",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "node_name":"127.0.0.1:35990_",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core_node_name":"core_node1"} current state version: 2
   [junit4]   2> 1524936 INFO  (OverseerStateUpdate-95418044618637316-127.0.0.1:35990_-n_0000000000) [n:127.0.0.1:35990_    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:35990",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "node_name":"127.0.0.1:35990_",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core_node_name":"core_node1"}
   [junit4]   2> 1524945 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_A56B512213E73DFE-001/shard-1-001/cores
   [junit4]   2> 1524945 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [    ] o.a.s.c.CoreContainer New CoreContainer 1591625572
   [junit4]   2> 1524945 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_A56B512213E73DFE-001/shard-1-001]
   [junit4]   2> 1524945 WARN  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_A56B512213E73DFE-001/shard-1-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_A56B512213E73DFE-001/shard-1-001/lib
   [junit4]   2> 1524946 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1524947 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1524947 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1524948 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1524948 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:52913/solr
   [junit4]   2> 1524948 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1524948 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1524948 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1524950 INFO  (zkCallback-1365-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2a296696 name:ZooKeeperConnection Watcher:127.0.0.1:52913 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1524950 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1524950 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1524952 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [n:127.0.0.1:51863_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1524953 INFO  (zkCallback-1366-thread-1-processing-n:127.0.0.1:51863_) [n:127.0.0.1:51863_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@440c08fc name:ZooKeeperConnection Watcher:127.0.0.1:52913/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1524953 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [n:127.0.0.1:51863_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1524961 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [n:127.0.0.1:51863_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1525038 INFO  (zkCallback-1366-thread-1-processing-n:127.0.0.1:51863_) [n:127.0.0.1:51863_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1525038 INFO  (zkCallback-1361-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1525038 INFO  (zkCallback-1359-thread-1-processing-n:127.0.0.1:35990_) [n:127.0.0.1:35990_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1525967 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [n:127.0.0.1:51863_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:51863_
   [junit4]   2> 1525967 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [n:127.0.0.1:51863_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:51863_
   [junit4]   2> 1525969 INFO  (zkCallback-1359-thread-1-processing-n:127.0.0.1:35990_) [n:127.0.0.1:35990_    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1525969 INFO  (zkCallback-1361-thread-1) [    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1525969 INFO  (zkCallback-1366-thread-1-processing-n:127.0.0.1:51863_) [n:127.0.0.1:51863_    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1525970 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [n:127.0.0.1:51863_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1525970 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [n:127.0.0.1:51863_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/95418044618637320-127.0.0.1:51863_-n_0000000001
   [junit4]   2> 1525971 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [n:127.0.0.1:51863_    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/95418044618637316-127.0.0.1:35990_-n_0000000000 to know if I could be the leader
   [junit4]   2> 1525982 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [n:127.0.0.1:51863_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1525982 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [n:127.0.0.1:51863_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1525983 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [n:127.0.0.1:51863_    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_A56B512213E73DFE-001/shard-1-001/cores
   [junit4]   2> 1525984 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [n:127.0.0.1:51863_    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {collection=collection1, configSetProperties=configsetprops.json, name=collection1, schema=schema.xml, coreNodeName=, loadOnStartup=true, dataDir=data/, shard=, config=solrconfig.xml, transient=false}
   [junit4]   2> 1525984 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [n:127.0.0.1:51863_    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_A56B512213E73DFE-001/shard-1-001/cores/collection1
   [junit4]   2> 1525984 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [n:127.0.0.1:51863_    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 1525985 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [n:127.0.0.1:51863_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 1525985 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[A56B512213E73DFE]) [n:127.0.0.1:51863_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1525985 INFO  (coreLoadExecutor-3720-thread-1-processing-n:127.0.0.1:51863_) [n:127.0.0.1:51863_ c:collection1   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 1525985 INFO  (coreLoadExecutor-3720-thread-1-processing-n:127.0.0.1:51863_) [n:127.0.0.1:51863_ c:collection1   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1525986 INFO  (coreLoadExecutor-3720-thread-1-processing-n:127.0.0.1:51863_) [n:127.0.0.1:51863_ c:collection1   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 1525986 INFO  (OverseerStateUpdate-95418044618637316-127.0.0.1:35990_-n_0000000000) [n:127.0.0.1:35990_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "shard":null,
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:51863",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "node_name":"127.0.0.1:51863_",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "operation":"state"} current state version: 3
   [junit4]   2> 1525987 INFO  (OverseerStateUpdate-95418044618637316-127.0.0.1:35990_-n_0000000000) [n:127.0.0.1:35990_    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "shard":null,
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:51863",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "node_name":"127.0.0.1:51863_",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 1525987 INFO  (OverseerStateUpdate-95418044618637316-127.0.0.1:35990_-n_0000000000) [n:127.0.0.1:35990_    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 1525987 INFO  (OverseerStateUpdate-95418044618637316-127.0.0.1:35990_-n_0000000000) [n:127.0.0.1:35990_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2
   [junit4]   2> 1526089 INFO  (zkCallback-1359-thread-1-processing-n:127.0.0.1:35990_) [n:127.0.0.1:35990_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1526090 INFO  (zkCallback-1361-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1526089 INFO  (zkCallback-1366-thread-1-processing-n:127.0.0.1:51863_) [n:127.0.0.1:51863_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1526987 INFO  (coreLoadExecutor-3720-thread-1-processing-n:127.0.0.1:51863_) [n:127.0.0.1:51863_ c:collection1   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1526987 INFO  (coreLoadExecutor-3720-thread-1-processing-n:127.0.0.1:51863_) [n:127.0.0.1:51863_ c:collection1   x:collection1] o.a.s.c.ZkController Check for collection zkNode:collection1
   [junit4]   2> 1526987 INFO  (coreLoadExecutor-3720-thread-1-processing-n:127.0.0.1:51863_) [n:127.0.0.1:51863_ c:collection1   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 1526987 INFO  (coreLoadExecutor-3720-thread-1-processing-n:127.0.0.1:51863_) [n:127.0.0.1:51863_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/collection1]
   [junit4]   2> 1526988 INFO  (coreLoadExecutor-3720-thread-1-processing-n:127.0.0.1:51863_) [n:127.0.0.1:51863_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/collection1] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 1526988 INFO  (coreLoadExecutor-3720-thread-1-processing-n:127.0.0.1:51863_) [n:127.0.0.1:51863_ c:collection1   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_A56B512213E73DFE-001/shard-1-001/cores/collection1'
   [junit4]   2> 1526988 INFO  (coreLoadExecutor-3720-thread-1-processing-n:127.0.0.1:51863_) [n:127.0.0.1:51863_ c:collection1   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1526988 INFO  (coreLoadExecutor-3720-thread-1-processing-n:127.0.0.1:51863_) [n:127.0.0.1:51863_ c:collection1   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1526990 INFO  (coreLoadExecutor-3720-thread-1-processing-n:127.0.0.1:51863_) [n:127.0.0.1:51863_ c:collection1   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 1526998 INFO  (coreLoadExecutor-3720-thread-1-processing-n:127.0.0.1:51863_) [n:127.0.0.1:51863_ c:collection1   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 1527020 INFO  (coreLoadExecutor-3720-thread-1-processing-n:127.0.0.1:51863_) [n:127.0.0.1:51863_ c:collection1   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.5.0
   [junit4]   2> 1527060 INFO  (coreLoadExecutor-3720-thread-1-processing-n:127.0.0.1:51863_) [n:127.0.0.1:51863_ c:collection1   x:collection1] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1527068 INFO  (coreLoadExecutor-3720-thread-1-processing-n:127.0.0.1:51863_) [n:127.0.0.1:51863_ c:collection1   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1527304 INFO  (coreLoadExecutor-3720-thread-1-processing-n:127.0.0.1:51863_) [n:127.0.0.1:51863_ c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 1527313 INFO  (coreLoadExecutor-3720-thread-1-processing-n:127.0.0.1:51863_) [n:127.0.0.1:51863_ c:collection1   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 1527315 INFO  (coreLoadExecutor-3720-thread-1-processing-n:127.0.0.1:51863_) [n:127.0.0.1:51863_ c:collection1   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 1527324 INFO  (coreLoadExecutor-3720-thread-1-processing-n:127.0.0.1:51863_) [n:127.0.0.1:51863_ c:collection1   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1527327 INFO  (coreLoadExecutor-3720-thread-1-processing-n:127.0.0.1:51863_) [n:127.0.0.1:51863_ c:collection1   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1527330 INFO  (coreLoadExecutor-3720-thread-1-processing-n:127.0.0.1:51863_) [n:127.0.0.1:51863_ c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1527330 INFO  (coreLoadExecutor-3720-thread-1-processing-n:127.0.0.1:51863_) [n:127.0.0.1:51863_ c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1527332 INFO  (coreLoadExecutor-3720-thread-1-processing-n:127.0.0.1:51863_) [n:127.0.0.1:51863_ c:collection1   x:collection1] o.a.s.c.ConfigSet

[...truncated too long message...]

enkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-ldif-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-mavibot-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-protocol-kerberos-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-protocol-ldap-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-protocol-shared-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-xdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/api-all-1.0.0-M20.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/bcprov-jdk15-1.45.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/cglib-nodep-2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/commons-collections-3.2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/commons-math3-3.4.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/easymock-3.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/ehcache-core-2.4.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/hadoop-common-2.6.0-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/hadoop-hdfs-2.6.0-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/hadoop-minikdc-2.6.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jackson-annotations-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jackson-databind-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jersey-core-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jersey-server-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jetty-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jetty-util-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/mina-core-2.0.0-M5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/objenesis-1.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/analysis/icu/lucene-analyzers-icu-5.5.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-analysis-extras/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/queryparser/classes/test:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/contrib/analysis-extras/lib/icu4j-54.1.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-launcher.jar:/x1/jenkins/.ant/lib/ivy-2.3.0.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-antlr.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-regexp.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-resolver.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-net.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jmf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-oro.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-xalan2.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-logging.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jsch.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit4.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-javamail.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jai.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jdepend.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bsf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-swing.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-netrexx.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bcel.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-testutil.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-log4j.jar:/x1/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.7/lib/tools.jar:/x1/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.3.2.jar com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J2-20160220_095809_482.events @/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J2-20160220_095809_482.suites -stdin
   [junit4] ERROR: JVM J2 ended with an exception: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1501)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:121)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:955)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:952)
   [junit4] 	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4] 	at java.lang.Thread.run(Thread.java:745)

BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:757: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:694: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:534: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1477: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1033: At least one slave process threw an exception, first: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.

Total time: 215 minutes 17 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
ERROR: Failed to archive {solr/build/solr-core/test/temp/junit4-J1-20160220_095809_481.events=solr/build/solr-core/test/temp/junit4-J1-20160220_095809_481.events, README.txt=README.txt, solr/build/solr-core/test/temp/junit4-J0-20160220_095809_481.events=solr/build/solr-core/test/temp/junit4-J0-20160220_095809_481.events, heapdumps/java_pid20839.hprof=heapdumps/java_pid20839.hprof, solr/build/solr-core/test/temp/junit4-J2-20160220_095809_482.events=solr/build/solr-core/test/temp/junit4-J2-20160220_095809_482.events} due to internal error; falling back to full archiving
java.lang.NullPointerException
	at jsync.protocol.ChangeInputStream.read(ChangeInputStream.java:69)
	at com.cloudbees.jenkins.plugins.jsync.archiver.MD5DigestingInputStream.read(MD5DigestingInputStream.java:39)
	at com.google.common.io.LimitInputStream.read(LimitInputStream.java:79)
	at java.io.FilterInputStream.read(FilterInputStream.java:107)
	at com.google.common.io.ByteStreams.copy(ByteStreams.java:193)
	at jsync.protocol.FileSequenceReader.read(FileSequenceReader.java:35)
	at com.cloudbees.jenkins.plugins.jsync.archiver.JSyncArtifactManager.remoteSync(JSyncArtifactManager.java:148)
	at com.cloudbees.jenkins.plugins.jsync.archiver.JSyncArtifactManager.archive(JSyncArtifactManager.java:72)
	at hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:219)
	at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:74)
	at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
	at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:776)
	at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:723)
	at hudson.model.Build$BuildExecution.post2(Build.java:183)
	at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:670)
	at hudson.model.Run.execute(Run.java:1763)
	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
	at hudson.model.ResourceController.execute(ResourceController.java:98)
	at hudson.model.Executor.run(Executor.java:381)
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1101 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1101/

3 tests failed.
FAILED:  org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=25279, name=collection5, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=25279, name=collection5, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]
	at __randomizedtesting.SeedInfo.seed([811581C0E9C440B0:941BE1A47382D48]:0)
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:54423/c_/l: collection already exists: awholynewstresscollection_collection5_0
	at __randomizedtesting.SeedInfo.seed([811581C0E9C440B0]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:576)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:240)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:229)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1573)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1594)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:960)


FAILED:  org.apache.solr.handler.TestReplicationHandler.doTestReplicateAfterCoreReload

Error Message:
expected:<[{indexVersion=1455845237037,generation=2,filelist=[_77.fdt, _77.fdx, _77.fnm, _77.nvd, _77.nvm, _77.si, _77_Direct_0.doc, _77_Direct_0.tim, _77_Direct_0.tip, _ee.cfe, _ee.cfs, _ee.si, _fl.cfe, _fl.cfs, _fl.si, _gs.cfe, _gs.cfs, _gs.si, _gz.cfe, _gz.cfs, _gz.si, _h6.cfe, _h6.cfs, _h6.si, _hc.cfe, _hc.cfs, _hc.si, _hd.cfe, _hd.cfs, _hd.si, _he.cfe, _he.cfs, _he.si, _hf.cfe, _hf.cfs, _hf.si, _hg.cfe, _hg.cfs, _hg.si, _hh.cfe, _hh.cfs, _hh.si, _hi.cfe, _hi.cfs, _hi.si, segments_2]}]> but was:<[{indexVersion=1455845237037,generation=3,filelist=[_77.fdt, _77.fdx, _77.fnm, _77.nvd, _77.nvm, _77.si, _77_Direct_0.doc, _77_Direct_0.tim, _77_Direct_0.tip, _ee.cfe, _ee.cfs, _ee.si, _fl.cfe, _fl.cfs, _fl.si, _gs.cfe, _gs.cfs, _gs.si, _gz.cfe, _gz.cfs, _gz.si, _h6.cfe, _h6.cfs, _h6.si, _hd.cfe, _hd.cfs, _hd.si, _hj.cfe, _hj.cfs, _hj.si, segments_3]}, {indexVersion=1455845237037,generation=2,filelist=[_77.fdt, _77.fdx, _77.fnm, _77.nvd, _77.nvm, _77.si, _77_Direct_0.doc, _77_Direct_0.tim, _77_Direct_0.tip, _ee.cfe, _ee.cfs, _ee.si, _fl.cfe, _fl.cfs, _fl.si, _gs.cfe, _gs.cfs, _gs.si, _gz.cfe, _gz.cfs, _gz.si, _h6.cfe, _h6.cfs, _h6.si, _hc.cfe, _hc.cfs, _hc.si, _hd.cfe, _hd.cfs, _hd.si, _he.cfe, _he.cfs, _he.si, _hf.cfe, _hf.cfs, _hf.si, _hg.cfe, _hg.cfs, _hg.si, _hh.cfe, _hh.cfs, _hh.si, _hi.cfe, _hi.cfs, _hi.si, segments_2]}]>

Stack Trace:
java.lang.AssertionError: expected:<[{indexVersion=1455845237037,generation=2,filelist=[_77.fdt, _77.fdx, _77.fnm, _77.nvd, _77.nvm, _77.si, _77_Direct_0.doc, _77_Direct_0.tim, _77_Direct_0.tip, _ee.cfe, _ee.cfs, _ee.si, _fl.cfe, _fl.cfs, _fl.si, _gs.cfe, _gs.cfs, _gs.si, _gz.cfe, _gz.cfs, _gz.si, _h6.cfe, _h6.cfs, _h6.si, _hc.cfe, _hc.cfs, _hc.si, _hd.cfe, _hd.cfs, _hd.si, _he.cfe, _he.cfs, _he.si, _hf.cfe, _hf.cfs, _hf.si, _hg.cfe, _hg.cfs, _hg.si, _hh.cfe, _hh.cfs, _hh.si, _hi.cfe, _hi.cfs, _hi.si, segments_2]}]> but was:<[{indexVersion=1455845237037,generation=3,filelist=[_77.fdt, _77.fdx, _77.fnm, _77.nvd, _77.nvm, _77.si, _77_Direct_0.doc, _77_Direct_0.tim, _77_Direct_0.tip, _ee.cfe, _ee.cfs, _ee.si, _fl.cfe, _fl.cfs, _fl.si, _gs.cfe, _gs.cfs, _gs.si, _gz.cfe, _gz.cfs, _gz.si, _h6.cfe, _h6.cfs, _h6.si, _hd.cfe, _hd.cfs, _hd.si, _hj.cfe, _hj.cfs, _hj.si, segments_3]}, {indexVersion=1455845237037,generation=2,filelist=[_77.fdt, _77.fdx, _77.fnm, _77.nvd, _77.nvm, _77.si, _77_Direct_0.doc, _77_Direct_0.tim, _77_Direct_0.tip, _ee.cfe, _ee.cfs, _ee.si, _fl.cfe, _fl.cfs, _fl.si, _gs.cfe, _gs.cfs, _gs.si, _gz.cfe, _gz.cfs, _gz.si, _h6.cfe, _h6.cfs, _h6.si, _hc.cfe, _hc.cfs, _hc.si, _hd.cfe, _hd.cfs, _hd.si, _he.cfe, _he.cfs, _he.si, _hf.cfe, _hf.cfs, _hf.si, _hg.cfe, _hg.cfs, _hg.si, _hh.cfe, _hh.cfs, _hh.si, _hi.cfe, _hi.cfs, _hi.si, segments_2]}]>
	at __randomizedtesting.SeedInfo.seed([811581C0E9C440B0:A4C29AF0998C4EB3]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.failNotEquals(Assert.java:647)
	at org.junit.Assert.assertEquals(Assert.java:128)
	at org.junit.Assert.assertEquals(Assert.java:147)
	at org.apache.solr.handler.TestReplicationHandler.doTestReplicateAfterCoreReload(TestReplicationHandler.java:1140)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Timeout occured while waiting response from server at: http://127.0.0.1:40004

Stack Trace:
org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:40004
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:585)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:240)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:229)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.makeRequest(CollectionsAPIDistributedZkTest.java:375)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testErrorHandling(CollectionsAPIDistributedZkTest.java:392)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test(CollectionsAPIDistributedZkTest.java:162)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:964)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:939)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.net.SocketTimeoutException: Read timed out
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.read(SocketInputStream.java:152)
	at java.net.SocketInputStream.read(SocketInputStream.java:122)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:160)
	at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:84)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:273)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
	at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
	at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)
	at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)
	at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197)
	at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)
	at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)
	at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685)
	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487)
	at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:481)
	... 47 more




Build Log:
[...truncated 11529 lines...]
   [junit4] Suite: org.apache.solr.handler.TestReplicationHandler
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_811581C0E9C440B0-001/init-core-data-001
   [junit4]   2> 1341110 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[811581C0E9C440B0]) [    ] o.a.s.SolrTestCaseJ4 ###Starting doTestStopPoll
   [junit4]   2> 1341111 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[811581C0E9C440B0]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_811581C0E9C440B0-001/solr-instance-001/collection1
   [junit4]   2> 1341116 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[811581C0E9C440B0]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 1341117 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[811581C0E9C440B0]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@2ccb08ff{/solr,null,AVAILABLE}
   [junit4]   2> 1341119 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[811581C0E9C440B0]) [    ] o.e.j.s.ServerConnector Started ServerConnector@263b9742{HTTP/1.1}{127.0.0.1:50011}
   [junit4]   2> 1341119 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[811581C0E9C440B0]) [    ] o.e.j.s.Server Started @1343851ms
   [junit4]   2> 1341119 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[811581C0E9C440B0]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_811581C0E9C440B0-001/solr-instance-001/collection1/data, hostPort=50011, hostContext=/solr}
   [junit4]   2> 1341119 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[811581C0E9C440B0]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@7b3cb2c6
   [junit4]   2> 1341119 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[811581C0E9C440B0]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_811581C0E9C440B0-001/solr-instance-001'
   [junit4]   2> 1341120 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[811581C0E9C440B0]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1341120 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[811581C0E9C440B0]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1341120 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[811581C0E9C440B0]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_811581C0E9C440B0-001/solr-instance-001/solr.xml
   [junit4]   2> 1341131 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[811581C0E9C440B0]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_811581C0E9C440B0-001/solr-instance-001/.
   [junit4]   2> 1341132 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[811581C0E9C440B0]) [    ] o.a.s.c.CoreContainer New CoreContainer 47820063
   [junit4]   2> 1341132 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[811581C0E9C440B0]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_811581C0E9C440B0-001/solr-instance-001]
   [junit4]   2> 1341132 WARN  (TEST-TestReplicationHandler.doTestStopPoll-seed#[811581C0E9C440B0]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_811581C0E9C440B0-001/solr-instance-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_811581C0E9C440B0-001/solr-instance-001/lib
   [junit4]   2> 1341132 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[811581C0E9C440B0]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1341133 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[811581C0E9C440B0]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1341133 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[811581C0E9C440B0]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1341133 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[811581C0E9C440B0]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1341133 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[811581C0E9C440B0]) [    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1341133 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[811581C0E9C440B0]) [    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1341134 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[811581C0E9C440B0]) [    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_811581C0E9C440B0-001/solr-instance-001/.
   [junit4]   2> 1341135 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[811581C0E9C440B0]) [    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {loadOnStartup=true, dataDir=data/, config=solrconfig.xml, transient=false, configSetProperties=configsetprops.json, schema=schema.xml, name=collection1}
   [junit4]   2> 1341135 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[811581C0E9C440B0]) [    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_811581C0E9C440B0-001/solr-instance-001/./collection1
   [junit4]   2> 1341135 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[811581C0E9C440B0]) [    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 1341136 INFO  (coreLoadExecutor-980-thread-1) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_811581C0E9C440B0-001/solr-instance-001/collection1'
   [junit4]   2> 1341139 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[811581C0E9C440B0]) [    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0
   [junit4]   2> 1341139 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[811581C0E9C440B0]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1341139 INFO  (coreLoadExecutor-980-thread-1) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1341140 INFO  (coreLoadExecutor-980-thread-1) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1341148 INFO  (coreLoadExecutor-980-thread-1) [    ] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 1341161 INFO  (coreLoadExecutor-980-thread-1) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.5.0
   [junit4]   2> 1341185 INFO  (coreLoadExecutor-980-thread-1) [    ] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1341193 INFO  (coreLoadExecutor-980-thread-1) [    ] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1341206 INFO  (coreLoadExecutor-980-thread-1) [    ] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 1341208 INFO  (coreLoadExecutor-980-thread-1) [    ] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_811581C0E9C440B0-001/solr-instance-001/collection1'
   [junit4]   2> 1341208 INFO  (coreLoadExecutor-980-thread-1) [    ] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_811581C0E9C440B0-001/solr-instance-001/./collection1
   [junit4]   2> 1341209 INFO  (coreLoadExecutor-980-thread-1) [    x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 1341209 INFO  (coreLoadExecutor-980-thread-1) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_811581C0E9C440B0-001/solr-instance-001/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_811581C0E9C440B0-001/solr-instance-001/./collection1/data/]
   [junit4]   2> 1341209 INFO  (coreLoadExecutor-980-thread-1) [    x:collection1] o.a.s.c.SolrCore JMX monitoring not detected for core: collection1
   [junit4]   2> 1341209 INFO  (coreLoadExecutor-980-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_811581C0E9C440B0-001/solr-instance-001/./collection1/data
   [junit4]   2> 1341209 INFO  (coreLoadExecutor-980-thread-1) [    x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_811581C0E9C440B0-001/solr-instance-001/./collection1/data/index/
   [junit4]   2> 1341209 WARN  (coreLoadExecutor-980-thread-1) [    x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_811581C0E9C440B0-001/solr-instance-001/./collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 1341210 INFO  (coreLoadExecutor-980-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_811581C0E9C440B0-001/solr-instance-001/./collection1/data/index
   [junit4]   2> 1341211 INFO  (coreLoadExecutor-980-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=6, maxMergeAtOnceExplicit=3, maxMergedSegmentMB=1.791015625, floorSegmentMB=0.6943359375, forceMergeDeletesPctAllowed=17.876505917755928, segmentsPerTier=19.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 1341214 INFO  (coreLoadExecutor-980-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(SimpleFSDirectory@/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_811581C0E9C440B0-001/index-SimpleFSDirectory-001 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6b5bed1a),segFN=segments_1,generation=1}
   [junit4]   2> 1341214 INFO  (coreLoadExecutor-980-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 1341215 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [    x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_811581C0E9C440B0-001/solr-instance-001/./collection1/data/
   [junit4]   2> 1341217 INFO  (coreLoadExecutor-980-thread-1) [    x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 1341220 INFO  (coreLoadExecutor-980-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1341221 INFO  (coreLoadExecutor-980-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1341222 INFO  (coreLoadExecutor-980-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1341223 INFO  (coreLoadExecutor-980-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1341228 INFO  (coreLoadExecutor-980-thread-1) [    x:collection1] o.a.s.c.PluginBag requestHandler : 'lazy' created with startup=lazy 
   [junit4]   2> 1341228 INFO  (coreLoadExecutor-980-thread-1) [    x:collection1] o.a.s.c.RequestHandlers Registered paths: defaults,/admin/ping,/admin/properties,/schema,/admin/logging,/replication,/admin/threads,/admin/luke,/update/csv,/update/json,/config,/get,/update,/admin/plugins,/admin/mbeans,lazy,/admin/file,/admin/system,standard,/admin/segments,/update/json/docs
   [junit4]   2> 1341229 INFO  (coreLoadExecutor-980-thread-1) [    x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 1341230 INFO  (coreLoadExecutor-980-thread-1) [    x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1341230 INFO  (coreLoadExecutor-980-thread-1) [    x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1341230 INFO  (coreLoadExecutor-980-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=6, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.7673005597787129]
   [junit4]   2> 1341233 INFO  (coreLoadExecutor-980-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(SimpleFSDirectory@/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_811581C0E9C440B0-001/index-SimpleFSDirectory-001 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6b5bed1a),segFN=segments_1,generation=1}
   [junit4]   2> 1341233 INFO  (coreLoadExecutor-980-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 1341233 INFO  (coreLoadExecutor-980-thread-1) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@361fe7a8[collection1] main
   [junit4]   2> 1341233 INFO  (coreLoadExecutor-980-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_811581C0E9C440B0-001/solr-instance-001/collection1/conf
   [junit4]   2> 1341233 INFO  (coreLoadExecutor-980-thread-1) [    x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {storageDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_811581C0E9C440B0-001/solr-instance-001/collection1/conf}
   [junit4]   2> 1341234 INFO  (coreLoadExecutor-980-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using file:dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_811581C0E9C440B0-001/solr-instance-001/collection1/conf
   [junit4]   2> 1341234 INFO  (coreLoadExecutor-980-thread-1) [    x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 1341234 INFO  (coreLoadExecutor-980-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler Replication enabled for following config files: schema.xml,xslt/dummy.xsl
   [junit4]   2> 1341234 INFO  (coreLoadExecutor-980-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1341234 INFO  (searcherExecutor-981-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@361fe7a8[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1341234 INFO  (coreLoadExecutor-980-thread-1) [    x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 1341641 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[811581C0E9C440B0]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_811581C0E9C440B0-001/solr-instance-002/collection1
   [junit4]   2> 1341645 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[811581C0E9C440B0]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 1341647 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[811581C0E9C440B0]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@659a5202{/solr,null,AVAILABLE}
   [junit4]   2> 1341647 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[811581C0E9C440B0]) [    ] o.e.j.s.ServerConnector Started ServerConnector@57e4feec{HTTP/1.1}{127.0.0.1:46134}
   [junit4]   2> 1341648 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[811581C0E9C440B0]) [    ] o.e.j.s.Server Started @1344379ms
   [junit4]   2> 1341648 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[811581C0E9C440B0]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_811581C0E9C440B0-001/solr-instance-002/collection1/data, hostPort=46134}
   [junit4]   2> 1341648 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[811581C0E9C440B0]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@7b3cb2c6
   [junit4]   2> 1341648 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[811581C0E9C440B0]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_811581C0E9C440B0-001/solr-instance-002'
   [junit4]   2> 1341648 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[811581C0E9C440B0]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1341648 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[811581C0E9C440B0]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1341648 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[811581C0E9C440B0]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_811581C0E9C440B0-001/solr-instance-002/solr.xml
   [junit4]   2> 1341663 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[811581C0E9C440B0]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_811581C0E9C440B0-001/solr-instance-002/.
   [junit4]   2> 1341663 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[811581C0E9C440B0]) [    ] o.a.s.c.CoreContainer New CoreContainer 1206750537
   [junit4]   2> 1341663 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[811581C0E9C440B0]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_811581C0E9C440B0-001/solr-instance-002]
   [junit4]   2> 1341663 WARN  (TEST-TestReplicationHandler.doTestStopPoll-seed#[811581C0E9C440B0]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_811581C0E9C440B0-001/solr-instance-002/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_811581C0E9C440B0-001/solr-instance-002/lib
   [junit4]   2> 1341664 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[811581C0E9C440B0]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1341665 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[811581C0E9C440B0]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1341665 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[811581C0E9C440B0]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1341665 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[811581C0E9C440B0]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1341665 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[811581C0E9C440B0]) [    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1341665 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[811581C0E9C440B0]) [    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1341666 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[811581C0E9C440B0]) [    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_811581C0E9C440B0-001/solr-instance-002/.
   [junit4]   2> 1341667 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[811581C0E9C440B0]) [    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {transient=false, dataDir=data/, name=collection1, loadOnStartup=true, configSetProperties=configsetprops.json, schema=schema.xml, config=solrconfig.xml}
   [junit4]   2> 1341667 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[811581C0E9C440B0]) [    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_811581C0E9C440B0-001/solr-instance-002/./collection1
   [junit4]   2> 1341667 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[811581C0E9C440B0]) [    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 1341668 INFO  (coreLoadExecutor-990-thread-1) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_811581C0E9C440B0-001/solr-instance-002/collection1'
   [junit4]   2> 1341669 INFO  (coreLoadExecutor-990-thread-1) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1341669 INFO  (coreLoadExecutor-990-thread-1) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1341668 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[811581C0E9C440B0]) [    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0
   [junit4]   2> 1341671 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[811581C0E9C440B0]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1341679 INFO  (coreLoadExecutor-990-thread-1) [    ] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 1341692 INFO  (coreLoadExecutor-990-thread-1) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.5.0
   [junit4]   2> 1341716 INFO  (coreLoadExecutor-990-thread-1) [    ] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1341720 INFO  (coreLoadExecutor-990-thread-1) [    ] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1341728 INFO  (coreLoadExecutor-990-thread-1) [    ] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 1341730 INFO  (coreLoadExecutor-990-thread-1) [    ] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_811581C0E9C440B0-001/solr-instance-002/collection1'
   [junit4]   2> 1341730 INFO  (coreLoadExecutor-990-thread-1) [    ] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_811581C0E9C440B0-001/solr-instance-002/./collection1
   [junit4]   2> 1341730 INFO  (coreLoadExecutor-990-thread-1) [    x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 1341730 INFO  (coreLoadExecutor-990-thread-1) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_811581C0E9C440B0-001/solr-instance-002/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_811581C0E9C440B0-001/solr-instance-002/./collection1/data/]
   [junit4]   2> 1341730 INFO  (coreLoadExecutor-990-thread-1) [    x:collection1] o.a.s.c.SolrCore JMX monitoring not detected for core: collection1
   [junit4]   2> 1341730 INFO  (coreLoadExecutor-990-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_811581C0E9C440B0-001/solr-instance-002/./collection1/data
   [junit4]   2> 1341731 INFO  (coreLoadExecutor-990-thread-1) [    x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_811581C0E9C440B0-001/solr-instance-002/./collection1/data/index/
   [junit4]   2> 1341731 WARN  (coreLoadExecutor-990-thread-1) [    x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_811581C0E9C440B0-001/solr-instance-002/./collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 1341732 INFO  (coreLoadExecutor-990-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_811581C0E9C440B0-001/solr-instance-002/./collection1/data/index
   [junit4]   2> 1341732 INFO  (coreLoadExecutor-990-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=6, maxMergeAtOnceExplicit=3, maxMergedSegmentMB=1.791015625, floorSegmentMB=0.6943359375, forceMergeDeletesPctAllowed=17.876505917755928, segmentsPerTier=19.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 1341737 INFO  (coreLoadExecutor-990-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(SimpleFSDirectory@/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_811581C0E9C440B0-001/index-SimpleFSDirectory-002 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6b5bed1a),segFN=segments_1,generation=1}
   [junit4]   2> 1341737 INFO  (coreLoadExecutor-990-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 1341737 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [    x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_811581C0E9C440B0-001/solr-instance-002/./collection1/data/
   [junit4]   2> 1341740 INFO  (coreLoadExecutor-990-thread-1) [    x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 1341742 INFO  (coreLoadExecutor-990-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1341743 INFO  (coreLoadExecutor-990-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1341744 INFO  (coreLoadExecutor-990-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1341746 INFO  (coreLoadExecutor-990-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1341747 INFO  (coreLoadExecutor-990-thread-1) [    x:collection1] o.a.s.c.PluginBag requestHandler : 'lazy' created with startup=lazy 
   [junit4]   2> 1341747 INFO  (coreLoadExecutor-990-thread-1) [    x:collection1] o.a.s.c.RequestHandlers Registered paths: defaults,/admin/ping,/admin/properties,/schema,/admin/logging,/replication,/admin/threads,/admin/luke,/update/csv,/update/json,/config,/get,/update,/admin/plugins,/admin/mbeans,lazy,/admin/file,/admin/system,standard,/admin/segments,/update/json/docs
   [junit4]   2> 1341748 INFO  (coreLoadExecutor-990-thread-1) [    x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 1341749 INFO  (coreLoadExecutor-990-thread-1) [    x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1341749 INFO  (coreLoadExecutor-990-thread-1) [    x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1341749 INFO  (coreLoadExecutor-990-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=6, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.7673005597787129]
   [junit4]   2> 1341750 INFO  (coreLoadExecutor-990-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(SimpleFSDirectory@/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_811581C0E9C440B0-001/index-SimpleFSDirectory-002 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6b5bed1a),segFN=segments_1,generation=1}
   [junit4]   2> 1341750 INFO  (coreLoadExecutor-990-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 1341750 INFO  (coreLoadExecutor-990-thread-1) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@8b7661d[collection1] main
   [junit4]   2> 1341751 INFO  (coreLoadExecutor-990-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_811581C0E9C440B0-001/solr-instance-002/collection1/conf
   [junit4]   2> 1341751 INFO  (coreLoadExecutor-990-thread-1) [    x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {storageDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_811581C0E9C440B0-001/solr-instance-002/collection1/conf}
   [junit4]   2> 1341751 INFO  (coreLoadExecutor-990-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using file:dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_811581C0E9C440B0-001/solr-instance-002/collection1/conf
   [junit4]   2> 1341751 INFO  (coreLoadExecutor-990-thread-1) [    x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 1341753 INFO  (coreLoadExecutor-990-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler Poll scheduled at an interval of 1000ms
   [junit4]   2> 1341753 INFO  (searcherExecutor-991-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@8b7661d[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1341753 INFO  (coreLoadExecutor-990-thread-1) [    x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 1341896 INFO  (qtp592038502-7845) [    x:collection1] o.a.s.c.S.Request [collection1]  webapp=/solr path=/replication params={wt=javabin&qt=/replication&command=indexversion&version=2} status=0 QTime=0
   [junit4]   2> 1342173 INFO  (qtp592038502-7846) [    x:collection1] o.a.s.c.S.Request [collection1]  webapp=/solr path=/select params={wt=javabin&version=2&sort=id+desc&q=*:*} hits=0 status=0 QTime=0
   [junit4]   2> 1342177 INFO  (qtp592038502-7847) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[0]} 0 1
   [junit4]   2> 1342179 INFO  (qtp592038502-7848) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[1]} 0 0
   [junit4]   2> 1342181 INFO  (qtp592038502-7849) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[2]} 0 0
   [junit4]   2> 1342182 INFO  (qtp592038502-7845) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[3]} 0 0
   [junit4]   2> 1342184 INFO  (qtp592038502-7846) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[4]} 0 0
   [junit4]   2> 1342186 INFO  (qtp592038502-7847) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[5]} 0 0
   [junit4]   2> 1342187 INFO  (qtp592038502-7848) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[6]} 0 0
   [junit4]   2> 1342189 INFO  (qtp592038502-7849) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[7]} 0 0
   [junit4]   2> 1342190 INFO  (qtp592038502-7845) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[8]} 0 0
   [junit4]   2> 1342192 INFO  (qtp592038502-7846) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[9]} 0 0
   [junit4]   2> 1342194 INFO  (qtp592038502-7847) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[10]} 0 0
   [junit4]   2> 1342196 INFO  (qtp592038502-7848) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[11]} 0 0
   [junit4]   2> 1342198 INFO  (qtp592038502-7849) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[12]} 0 0
   [junit4]   2> 1342200 INFO  (qtp592038502-7845) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[13]} 0 0
   [junit4]   2> 1342201 INFO  (qtp592038502-7846) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[14]} 0 0
   [junit4]   2> 1342203 INFO  (qtp592038502-7847) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[15]} 0 0
   [junit4]   2> 1342205 INFO  (qtp592038502-7848) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[16]} 0 0
   [junit4]   2> 1342206 INFO  (qtp592038502-7849) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[17]} 0 0
   [junit4]   2> 1342208 INFO  (qtp592038502-7845) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[18]} 0 0
   [junit4]   2> 1342210 INFO  (qtp592038502-7846) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[19]} 0 0
   [junit4]   2> 1342212 INFO  (qtp592038502-7847) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[20]} 0 0
   [junit4]   2> 1342214 INFO  (qtp592038502-7848) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[21]} 0 0
   [junit4]   2> 1342216 INFO  (qtp592038502-7849) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[22]} 0 0
   [junit4]   2> 1342218 INFO  (qtp592038502-7845) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[23]} 0 0
   [junit4]   2> 1342220 INFO  (qtp592038502-7846) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[24]} 0 0
   [junit4]   2> 1342222 INFO  (qtp592038502-7847) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[25]} 0 0
   [junit4]   2> 1342224 INFO  (qtp592038502-7848) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[26]} 0 0
   [junit4]   2> 1342226 INFO  (qtp592038502-7849) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[27]} 0 0
   [junit4]   2> 1342228 INFO  (qtp592038502-7845) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[28]} 0 0
   [junit4]   2> 1342230 INFO  (qtp592038502-7846) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[29]} 0 0
   [junit4]   2> 1342232 INFO  (qtp592038502-7847) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[30]} 0 0
   [junit4]   2> 1342234 INFO  (qtp592038502-7848) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[31]} 0 0
   [junit4]   2> 1342237 INFO  (qtp592038502-7849) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[32]} 0 0
   [junit4]   2> 1342239 INFO  (qtp592038502-7845) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[33]} 0 0
   [junit4]   2> 1342241 INFO  (qtp592038502-7846) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[34]} 0 0
   [junit4]   2> 1342243 INFO  (qtp592038502-7847) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[35]} 0 0
   [junit4]   2> 1342245 INFO  (qtp592038502-7848) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[36]} 0 0
   [junit4]   2> 1342247 INFO  (qtp592038502-7849) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[37]} 0 0
   [junit4]   2> 1342249 INFO  (qtp592038502-7845) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[38]} 0 0
   [junit4]   2> 1342251 INFO  (qtp592038502-7846) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[39]} 0 0
   [junit4]   2> 1342253 INFO  (qtp592038502-7847) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[40]} 0 0
   [junit4]   2> 1342255 INFO  (qtp592038502-7848) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[41]} 0 0
   [junit4]   2> 1342257 INFO  (qtp592038502-7849) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[42]} 0 0
   [junit4]   2> 1342259 INFO  (qtp592038502-7845) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[43]} 0 0
   [junit4]   2> 1342261 INFO  (qtp592038502-7846) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[44]} 0 0
   [junit4]   2> 1342263 INFO  (qtp592038502-7847) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[45]} 0 0
   [junit4]   2> 1342266 INFO  (qtp592038502-7848) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[46]} 0 0
   [junit4]   2> 1342267 INFO  (qtp592038502-7849) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[47]} 0 0
   [junit4]   2> 1342269 INFO  (qtp592038502-7845) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[48]} 0 0
   [junit4]   2> 1342271 INFO  (qtp592038502-7846) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[49]} 0 0
   [junit4]   2> 1342273 INFO  (qtp592038502-7847) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[50]} 0 0
   [junit4]   2> 1342275 INFO  (qtp592038502-7848) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[51]} 0 0
   [junit4]   2> 1342277 INFO  (qtp592038502-7849) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[52]} 0 0
   [junit4]   2> 1342279 INFO  (qtp592038502-7845) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[53]} 0 0
   [junit4]   2> 1342281 INFO  (qtp592038502-7846) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[54]} 0 0
   [junit4]   2> 1342283 INFO  (qtp592038502-7847) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[55]} 0 0
   [junit4]   2> 1342285 INFO  (qtp592038502-7848) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[56]} 0 0
   [junit4]   2> 1342287 INFO  (qtp592038502-7849) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[57]} 0 0
   [junit4]   2> 1342289 INFO  (qtp592038502-7845) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[58]} 0 0
   [junit4]   2> 1342291 INFO  (qtp592038502-7846) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[59]} 0 0
   [junit4]   2> 1342292 INFO  (qtp592038502-7847) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[60]} 0 0
   [junit4]   2> 1342294 INFO  (qtp592038502-7848) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[61]} 0 0
   [junit4]   2> 1342296 INFO  (qtp592038502-7849) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[62]} 0 0
   [junit4]   2> 1342298 INFO  (qtp592038502-7845) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[63]} 0 0
   [junit4]   2> 1342300 INFO  (qtp592038502-7846) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[64]} 0 0
   [junit4]   2> 1342302 INFO  (qtp592038502-7847) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[65]} 0 0
   [junit4]   2> 1342304 INFO  (qtp592038502-7848) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[66]} 0 0
   [junit4]   2> 1342306 INFO  (qtp592038502-7849) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[67]} 0 0
   [junit4]   2> 1342309 INFO  (qtp592038502-7845) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[68]} 0 0
   [junit4]   2> 1342311 INFO  (qtp592038502-7846) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[69]} 0 0
   [junit4]   2> 1342313 INFO  (qtp592038502-7847) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[70]} 0 0
   [junit4]   2> 1342315 INFO  (qtp592038502-7848) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[71]} 0 0
   [junit4]   2> 1342317 INFO  (qtp592038502-7849) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[72]} 0 0
   [junit4]   2> 1342319 INFO  (qtp592038502-7845) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[73]} 0 0
   [junit4]   2> 1342321 INFO  (qtp592038502-7846) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[74]} 0 0
   [junit4]   2> 1342323 INFO  (qtp592038502-7847) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[75]} 0 0
   [junit4]   2> 1342325 INFO  (qtp592038502-7848) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[76]} 0 0
   [junit4]   2> 1342327 INFO  (qtp592038502-7849) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[77]} 0 0
   [junit4]   2> 1342329 INFO  (qtp592038502-7845) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[78]} 0 0
   [junit4]   2> 1342331 INFO  (qtp592038502-7846) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[79]} 0 0
   [junit4]   2> 1342333 INFO  (qtp592038502-7847) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[80]} 0 0
   [junit4]   2> 1342335 INFO  (qtp592038502-7848) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[81]} 0 0
   [junit4]   2> 1342338 INFO  (qtp592038502-7849) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[82]} 0 0
   [junit4]   2> 1342340 INFO  (qtp592038502-7845) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[83]} 0 0
   [junit4]   2> 1342342 INFO  (qtp592038502-7846) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[84]} 0 0
   [junit4]   2> 1342344 INFO  (qtp592038502-7847) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[85]} 0 0
   [junit4]   2> 1342346 INFO  (qtp592038502-7848) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[86]} 0 0
   [junit4]   2> 1342348 INFO  (qtp592038502-7849) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[87]} 0 0
   [junit4]   2> 1342350 INFO  (qtp592038502-7845) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[88]} 0 0
   [junit4]   2> 1342352 INFO  (qtp592038502-7846) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[89]} 0 0
   [junit4]   2> 1342354 INFO  (qtp592038502-7847) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[90]} 0 0
   [junit4]   2> 1342356 INFO  (qtp592038502-7848) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[91]} 0 0
   [junit4]   2> 1342358 INFO  (qtp592038502-7849) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[92]} 0 0
   [junit4]   2> 1342360 INFO  (qtp592038502-7845) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[93]} 0 0
   [junit4]   2> 1342362 INFO  (qtp592038502-7846) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[94]} 0 0
   [junit4]   2> 1342364 INFO  (qtp592038502-7847) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[95]} 0 0
   [junit4]   2> 1342366 INFO  (qtp592038502-7848) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[96]} 0 0
   [junit4]   2> 1342368 INFO  (qtp592038502-7849) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[97]} 0 0
   [junit4]   2> 1342370 INFO  (qtp592038502-7845) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[98]} 0 0
   [junit4]   2> 1342372 INFO  (qtp592038502-7846) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[99]} 0 0
   [junit4]   2> 1342374 INFO  (qtp592038502-7847) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[100]} 0 0
   [junit4]   2> 1342376 INFO  (qtp592038502-7848) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[101]} 0 0
   [junit4]   2> 1342378 INFO  (qtp592038502-7849) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[102]} 0 0
   [junit4]   2> 1342380 INFO  (qtp592038502-7845) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[103]} 0 0
   [junit4]   2> 1342381 INFO  (qtp592038502-7846) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[104]} 0 0
   [junit4]   2> 1342383 INFO  (qtp592038502-7847) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[105]} 0 0
   [junit4]   2> 1342385 INFO  (qtp592038502-7848) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[106]} 0 0
   [junit4]   2> 1342387 INFO  (qtp592038502-7849) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[107]} 0 0
   [junit4]   2> 1342389 INFO  (qtp592038502-7845) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[108]} 0 0
   [junit4]   2> 1342391 INFO  (qtp592038502-7846) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[109]} 0 0
   [junit4]   2> 1342393 INFO  (qtp592038502-7847) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[110]} 0 0
   [junit4]   2> 1342395 INFO  (qtp592038502-7848) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[111]} 0 0
   [junit4]   2> 1342397 INFO  (qtp592038502-7849) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[112]} 0 0
   [junit4]   2> 1342398 INFO  (qtp592038502-7845) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[113]} 0 0
   [junit4]   2> 1342400 INFO  (qtp592038502-7846) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[114]} 0 0
   [junit4]   2> 1342402 INFO  (qtp592038502-7847) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[115]} 0 0
   [junit4]   2> 1342404 INFO  (qtp592038502-7848) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[116]} 0 0
   [junit4]   2> 1342406 INFO  (qtp592038502-7849) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[117]} 0 0
   [junit4]   2> 1342408 INFO  (qtp592038502-7845) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[118]} 0 0
   [junit4]   2> 1342410 INFO  (qtp592038502-7846) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[119]} 0 0
   [junit4]   2> 1342412 INFO  (qtp592038502-7847) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[120]} 0 0
   [junit4]   2> 1342413 INFO  (qtp592038502-7848) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[121]} 0 0
   [junit4]   2> 1342415 INFO  (qtp592038502-7849) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[122]} 0 0
   [junit4]   2> 1342417 INFO  (qtp592038502-7845) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[123]} 0 0
   [junit4]   2> 1342419 INFO  (qtp592038502-7846) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[124]} 0 0
   [junit4]   2> 1342420 INFO  (qtp592038502-7847) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[125]} 0 0
   [junit4]   2> 1342422 INFO  (qtp592038502-7848) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[126]} 0 0
   [junit4]   2> 1342424 INFO  (qtp592038502-7849) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[127]} 0 0
   [junit4]   2> 1342426 INFO  (qtp592038502-7845) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[128]} 0 0
   [junit4]   2> 1342428 INFO  (qtp592038502-7846) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[129]} 0 0
   [junit4]   2> 1342429 INFO  (qtp592038502-7847) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[130]} 0 0
   [junit4]   2> 1342431 INFO  (qtp592038502-7848) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[131]} 0 0
   [junit4]   2> 1342433 INFO  (qtp592038502-7849) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[132]} 0 0
   [junit4]   2> 1342435 INFO  (qtp592038502-7845) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[133]} 0 0
   [junit4]   2> 1342437 INFO  (qtp592038502-7846) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[134]} 0 0
   [junit4]   2> 1342439 INFO  (qtp592038502-7847) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[135]} 0 0
   [junit4]   2> 1342441 INFO  (qtp592038502-7848) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[136]} 0 0
   [junit4]   2> 1342442 INFO  (qtp592038502-7849) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[137]} 0 0
   [junit4]   2> 1342444 INFO  (qtp592038502-7845) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[138]} 0 0
   [junit4]   2> 1342446 INFO  (qtp592038502-7846) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[139]} 0 0
   [junit4]   2> 1342448 INFO  (qtp592038502-7847) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[140]} 0 0
   [junit4]   2> 1342450 INFO  (qtp592038502-7848) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[141]} 0 0
   [junit4]   2> 1342451 INFO  (qtp592038502-7849) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[142]} 0 0
   [junit4]   2> 1342453 INFO  (qtp592038502-7845) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[143]} 0 0
   [junit4]   2> 1342455 INFO  (qtp592038502-7846) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[144]} 0 0
   [junit4]   2> 1342457 INFO  (qtp592038502-7847) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[145]} 0 0
   [junit4]   2> 1342459 INFO  (qtp592038502-7848) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[146]} 0 0
   [junit4]   2> 1342460 INFO  (qtp592038502-7849) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[147]} 0 0
   [junit4]   2> 1342462 INFO  (qtp592038502-7845) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[148]} 0 0
   [junit4]   2> 1342464 INFO  (qtp592038502-7846) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[149]} 0 0
   [junit4]   2> 1342466 INFO  (qtp592038502-7847) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[150]} 0 0
   [junit4]   2> 1342468 INFO  (qtp592038502-7848) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[151]} 0 0
   [junit4]   2> 1342470 INFO  (qtp592038502-7849) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[152]} 0 0
   [junit4]   2> 1342471 INFO  (qtp592038502-7845) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[153]} 0 0
   [junit4]   2> 1342473 INFO  (qtp592038502-7846) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[154]} 0 0
   [junit4]   2> 1342475 INFO  (qtp592038502-7847) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[155]} 0 0
   [junit4]   2> 1342477 INFO  (qtp592038502-7848) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[156]} 0 0
   [junit4]   2> 1342479 INFO  (qtp592038502-7849) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[157]} 0 0
   [junit4]   2> 1342481 INFO  (qtp592038502-7845) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[158]} 0 0
   [junit4]   2> 1342483 INFO  (qtp592038502-7846) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[159]} 0 0
   [junit4]   2> 1342484 INFO  (qtp592038502-7847) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[160]} 0 0
   [junit4]   2> 1342486 INFO  (qtp592038502-7848) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[161]} 0 0
   [junit4]   2> 1342488 INFO  (qtp592038502-7849) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[162]} 0 0
   [junit4]   2> 1342490 INFO  (qtp592038502-7845) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[163]} 0 0
   [junit4]   2> 1342492 INFO  (qtp592038502-7846) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[164]} 0 0
   [junit4]   2> 1342494 INFO  (qtp592038502-7847) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[165]} 0 0
   [junit4]   2> 1342495 INFO  (qtp592038502-7848) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[166]} 0 0
   [junit4]   2> 1342497 INFO  (qtp592038502-7849) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[167]} 0 0
   [junit4]   2> 1342499 INFO  (qtp592038502-7845) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[168]} 0 0
   [junit4]   2> 1342501 INFO  (qtp592038502-7846) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[169]} 0 0
   [junit4]   2> 1342503 INFO  (qtp592038502-7847) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[170]} 0 0
   [junit4]   2> 1342505 INFO  (qtp592038502-7848) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[171]} 0 0
   [junit4]   2> 1342506 INFO  (qtp592038502-7849) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[172]} 0 0
   [junit4]   2> 1342508 INFO  (qtp592038502-7845) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[173]} 0 0
   [junit4]   2> 1342510 INFO  (qtp592038502-7846) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[174]} 0 0
   [junit4]   2> 1342512 INFO  (qtp592038502-7847) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[175]} 0 0
   [junit4]   2> 1342514 INFO  (qtp592038502-7848) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[176]} 0 0
   [junit4]   2> 1342515 INFO  (qtp592038502-7849) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[177]} 0 0
   [junit4]   2> 1342517 INFO  (qtp592038502-7845) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[178]} 0 0
   [junit4]   2> 1342519 INFO  (qtp592038502-7846) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[179]} 0 0
   [junit4]   2> 1342521 INFO  (qtp592038502-7847) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[180]} 0 0
   [junit4]   2> 1342523 INFO  (qtp592038502-7848) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[181]} 0 0
   [junit4]   2> 1342524 INFO  (qtp592038502-7849) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[182]} 0 0
   [junit4]   2> 1342526 INFO  (qtp592038502-7845) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[183]} 0 0
   [junit4]   2> 1342528 INFO  (qtp592038502-7846) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[184]} 0 0
   [junit4]   2> 1342530 INFO  (qtp592038502-7847) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[185]} 0 0
   [junit4]   2> 1342531 INFO  (qtp592038502-7848) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[186]} 0 0
   [junit4]   2> 1342533 INFO  (qtp592038502-7849) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[187]} 0 0
   [junit4]   2> 1342535 INFO  (qtp592038502-7845) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[188]} 0 0
   [junit4]   2> 1342537 INFO  (qtp592038502-7846) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[189]} 0 0
   [junit4]   2> 1342538 INFO  (qtp592038502-7847) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[190]} 0 0
   [junit4]   2> 1342540 INFO  (qtp592038502-7848) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[191]} 0 0
   [junit4]   2> 1342542 INFO  (qtp592038502-7849) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[192]} 0 0
   [junit4]   2> 1342544 INFO  (qtp592038502-7845) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[193]} 0 0
   [junit4]   2> 1342545 INFO  (qtp592038502-7846) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[194]} 0 0
   [junit4]   2> 1342547 INFO  (qtp592038502-7847) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[195]} 0 0
   [junit4]   2> 1342549 INFO  (qtp592038502-7848) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[196]} 0 0
   [junit4]   2> 1342551 INFO  (qtp592038502-7849) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[197]} 0 0
   [junit4]   2> 1342552 INFO  (qtp592038502-7845) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collect

[...truncated too long message...]

apter.java:172)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.getMBeanInfo(MetricsSourceAdapter.java:151)
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getClassName(DefaultMBeanServerInterceptor.java:1804)
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.safeGetClassName(DefaultMBeanServerInterceptor.java:1595)
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.checkMBeanPermission(DefaultMBeanServerInterceptor.java:1813)
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.exclusiveUnregisterMBean(DefaultMBeanServerInterceptor.java:430)
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.unregisterMBean(DefaultMBeanServerInterceptor.java:415)
   [junit4]   2> 	at com.sun.jmx.mbeanserver.JmxMBeanServer.unregisterMBean(JmxMBeanServer.java:546)
   [junit4]   2> 	at org.apache.hadoop.metrics2.util.MBeans.unregister(MBeans.java:81)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.stopMBeans(MetricsSourceAdapter.java:227)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.stop(MetricsSourceAdapter.java:212)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.stopSources(MetricsSystemImpl.java:461)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.stop(MetricsSystemImpl.java:212)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.shutdown(MetricsSystemImpl.java:592)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.shutdownInstance(DefaultMetricsSystem.java:72)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.shutdown(DefaultMetricsSystem.java:68)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.metrics.NameNodeMetrics.shutdown(NameNodeMetrics.java:145)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.NameNode.stop(NameNode.java:822)
   [junit4]   2> 	at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1720)
   [junit4]   2> 	at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1699)
   [junit4]   2> 	at org.apache.solr.cloud.hdfs.HdfsTestUtil.teardownClass(HdfsTestUtil.java:198)
   [junit4]   2> 	at org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.teardownClass(HdfsCollectionsAPIDistributedZkTest.java:52)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   [junit4]   2> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2> 	at java.lang.reflect.Method.invoke(Method.java:606)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:834)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> Caused by: java.lang.NullPointerException
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.blockmanagement.BlocksMap.size(BlocksMap.java:198)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.getTotalBlocks(BlockManager.java:3291)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlocksTotal(FSNamesystem.java:6223)
   [junit4]   2> 	... 54 more
   [junit4]   2> 3618247 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[811581C0E9C440B0]-worker) [    ] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> Feb 19, 2016 1:54:40 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 2 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene54): {range_facet_l=FST50, range_facet_i_dv=FST50, _version_=PostingsFormat(name=MockRandom), text=PostingsFormat(name=LuceneVarGapDocFreqInterval), other_tl1=PostingsFormat(name=MockRandom), intDefault=PostingsFormat(name=MockRandom), id=PostingsFormat(name=Asserting), a_i1=FST50, a_t=FST50, range_facet_l_dv=PostingsFormat(name=Asserting), multiDefault=FST50, timestamp=FST50}, docValues:{range_facet_i_dv=DocValuesFormat(name=Direct), range_facet_l_dv=DocValuesFormat(name=Asserting), timestamp=DocValuesFormat(name=Direct)}, sim=DefaultSimilarity, locale=sr-CS, timezone=Etc/UTC
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_80 (64-bit)/cpus=4,threads=2,free=115654488,total=525860864
   [junit4]   2> NOTE: All tests run in this JVM: [HighlighterTest, TestWordDelimiterFilterFactory, TestSortByMinMaxFunction, SolrIndexSplitterTest, DocumentBuilderTest, MigrateRouteKeyTest, TestUseDocValuesAsStored, SuggesterFSTTest, DOMUtilTest, CheckHdfsIndexTest, ZkSolrClientTest, SolrInfoMBeanTest, SortByFunctionTest, TestSchemaSimilarityResource, TestTrie, SpellPossibilityIteratorTest, TestCollationFieldDocValues, TestAuthenticationFramework, AnalyticsQueryTest, TestQuerySenderListener, TestPartialUpdateDeduplication, HdfsThreadLeakTest, TestManagedResourceStorage, AsyncMigrateRouteKeyTest, TestSurroundQueryParser, RollingRestartTest, BlockCacheTest, MoreLikeThisHandlerTest, TestJsonFacets, TestFieldResource, TermsComponentTest, TestInitParams, LoggingHandlerTest, HdfsDirectoryTest, SpellCheckComponentTest, TestDistributedGrouping, TimeZoneUtilsTest, TestRebalanceLeaders, TestConfigReload, TestExpandComponent, TestUpdate, SpatialFilterTest, ChaosMonkeySafeLeaderTest, MinimalSchemaTest, DistributedSuggestComponentTest, SimpleMLTQParserTest, TestExactStatsCache, ClusterStateUpdateTest, OpenExchangeRatesOrgProviderTest, BigEndianAscendingWordSerializerTest, TestSolrCLIRunExample, TestFastLRUCache, SolrPluginUtilsTest, TestNonDefinedSimilarityFactory, TriLevelCompositeIdRoutingTest, DistributedMLTComponentTest, TestInitQParser, TestLFUCache, TestSystemIdResolver, DistanceFunctionTest, TestSolrQueryResponse, TestBinaryResponseWriter, DeleteInactiveReplicaTest, AliasIntegrationTest, WordBreakSolrSpellCheckerTest, DocValuesTest, ConjunctionSolrSpellCheckerTest, TestConfigSetImmutable, TestSmileRequest, TestQueryUtils, DistributedTermsComponentTest, TestRequestStatusCollectionAPI, ShardSplitTest, SimplePostToolTest, TestSimpleTrackingShardHandler, CSVRequestHandlerTest, ChangedSchemaMergeTest, SpellingQueryConverterTest, TestNoOpRegenerator, BasicFunctionalityTest, TestDynamicFieldCollectionResource, TestManagedStopFilterFactory, ZkCLITest, TermVectorComponentTest, TestDocumentBuilder, HdfsCollectionsAPIDistributedZkTest]
   [junit4] Completed [286/566 (3!)] on J1 in 394.33s, 1 test, 1 error <<< FAILURES!

[...truncated 901 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:757: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:694: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:534: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1477: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1033: There were test failures: 566 suites (7 ignored), 2279 tests, 2 errors, 1 failure, 34 ignored (18 assumptions) [seed: 811581C0E9C440B0]

Total time: 212 minutes 4 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1100 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1100/

4 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.handler.TestReplicationHandler

Error Message:
ObjectTracker found 7 object(s) that were not released!!! [NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory]

Stack Trace:
java.lang.AssertionError: ObjectTracker found 7 object(s) that were not released!!! [NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory]
	at __randomizedtesting.SeedInfo.seed([3D49B6BCFBA4BC04]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.junit.Assert.assertNull(Assert.java:551)
	at org.apache.solr.SolrTestCaseJ4.afterClass(SolrTestCaseJ4.java:228)
	at sun.reflect.GeneratedMethodAccessor18.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:834)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.security.BasicAuthIntegrationTest.testBasics

Error Message:
Test abandoned because suite timeout was reached.

Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
	at __randomizedtesting.SeedInfo.seed([3D49B6BCFBA4BC04]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.security.BasicAuthIntegrationTest

Error Message:
Suite timeout exceeded (>= 7200000 msec).

Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
	at __randomizedtesting.SeedInfo.seed([3D49B6BCFBA4BC04]:0)


FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Timeout occured while waiting response from server at: http://127.0.0.1:56311/cu_i/fg

Stack Trace:
org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:56311/cu_i/fg
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:585)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:240)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:229)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.makeRequest(CollectionsAPIDistributedZkTest.java:375)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testErrorHandling(CollectionsAPIDistributedZkTest.java:502)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test(CollectionsAPIDistributedZkTest.java:162)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:964)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:939)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.net.SocketTimeoutException: Read timed out
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.read(SocketInputStream.java:152)
	at java.net.SocketInputStream.read(SocketInputStream.java:122)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:160)
	at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:84)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:273)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
	at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
	at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)
	at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)
	at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197)
	at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)
	at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)
	at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685)
	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487)
	at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:481)
	... 47 more




Build Log:
[...truncated 11345 lines...]
   [junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest
   [junit4]   2> 181034 ERROR (Finalizer) [    ] o.a.s.c.CoreContainer CoreContainer was not close prior to finalize(), indicates a bug -- POSSIBLE RESOURCE LEAK!!!  instance=2019256400
   [junit4]   2> 181034 ERROR (Finalizer) [    ] o.a.s.c.CoreContainer CoreContainer was not close prior to finalize(), indicates a bug -- POSSIBLE RESOURCE LEAK!!!  instance=1843133625
   [junit4]   2> 181034 ERROR (Finalizer) [    ] o.a.s.c.CoreContainer CoreContainer was not close prior to finalize(), indicates a bug -- POSSIBLE RESOURCE LEAK!!!  instance=710656256
   [junit4]   2> 181034 ERROR (Finalizer) [    ] o.a.s.c.CoreContainer CoreContainer was not close prior to finalize(), indicates a bug -- POSSIBLE RESOURCE LEAK!!!  instance=142892686
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_3D49B6BCFBA4BC04-001/init-core-data-001
   [junit4]   2> 181044 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[3D49B6BCFBA4BC04]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true)
   [junit4]   2> 181044 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[3D49B6BCFBA4BC04]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /cu_i/fg
   [junit4]   2> 181050 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 181050 INFO  (Thread-1267) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 181050 INFO  (Thread-1267) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 181150 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.ZkTestServer start zk server on port:52759
   [junit4]   2> 181151 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 181217 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 181228 INFO  (zkCallback-133-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1278256c name:ZooKeeperConnection Watcher:127.0.0.1:52759 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 181228 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 181228 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 181228 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 181242 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 181252 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 181253 INFO  (zkCallback-134-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@75f05481 name:ZooKeeperConnection Watcher:127.0.0.1:52759/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 181253 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 181253 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 181253 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 181256 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 181258 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 181259 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 181261 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 181261 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 181265 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 181266 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 181269 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 181269 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 181271 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 181271 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 181273 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 181273 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 181275 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 181275 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 181277 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 181277 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 181279 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 181279 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 181289 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 181289 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 181291 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 181291 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 181293 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 181293 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 181302 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 181304 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 181305 INFO  (zkCallback-135-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5477e0e8 name:ZooKeeperConnection Watcher:127.0.0.1:52759 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 181305 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 181305 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 181305 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 181308 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 181339 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 181340 INFO  (zkCallback-136-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2d6e6e7b name:ZooKeeperConnection Watcher:127.0.0.1:52759/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 181347 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 181347 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 181348 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf2/solrconfig.xml
   [junit4]   2> 181348 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/solrconfig.xml
   [junit4]   2> 181356 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf2/schema.xml
   [junit4]   2> 181356 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/schema.xml
   [junit4]   2> 181359 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf2/enumsConfig.xml
   [junit4]   2> 181359 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/enumsConfig.xml
   [junit4]   2> 181361 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf2/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 181361 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 181363 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf2/stopwords.txt
   [junit4]   2> 181363 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/stopwords.txt
   [junit4]   2> 181365 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf2/protwords.txt
   [junit4]   2> 181366 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/protwords.txt
   [junit4]   2> 181367 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf2/currency.xml
   [junit4]   2> 181368 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/currency.xml
   [junit4]   2> 181369 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf2/open-exchange-rates.json
   [junit4]   2> 181370 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/open-exchange-rates.json
   [junit4]   2> 181371 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf2/mapping-ISOLatin1Accent.txt
   [junit4]   2> 181371 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/mapping-ISOLatin1Accent.txt
   [junit4]   2> 181373 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf2/old_synonyms.txt
   [junit4]   2> 181373 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/old_synonyms.txt
   [junit4]   2> 181375 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf2/synonyms.txt
   [junit4]   2> 181375 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/synonyms.txt
   [junit4]   2> 181377 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/elevate.xml to /configs/conf2/elevate.xml
   [junit4]   2> 181377 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/elevate.xml
   [junit4]   2> 181533 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_3D49B6BCFBA4BC04-001/control-001/cores/collection1
   [junit4]   2> 181548 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 181573 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@153df5e5{/cu_i/fg,null,AVAILABLE}
   [junit4]   2> 181574 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.e.j.s.ServerConnector Started ServerConnector@16da9848{HTTP/1.1}{127.0.0.1:45254}
   [junit4]   2> 181574 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.e.j.s.Server Started @184649ms
   [junit4]   2> 181574 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_3D49B6BCFBA4BC04-001/tempDir-001/control/data, hostPort=45254, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_3D49B6BCFBA4BC04-001/control-001/cores, hostContext=/cu_i/fg}
   [junit4]   2> 181575 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@41692a49
   [junit4]   2> 181575 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_3D49B6BCFBA4BC04-001/control-001'
   [junit4]   2> 181575 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 181575 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 181576 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 181583 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 181596 INFO  (zkCallback-137-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@22f4d1c4 name:ZooKeeperConnection Watcher:127.0.0.1:52759/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 181597 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 181597 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 181613 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 181613 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_3D49B6BCFBA4BC04-001/control-001/solr.xml
   [junit4]   2> 181630 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_3D49B6BCFBA4BC04-001/control-001/cores
   [junit4]   2> 181630 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.CoreContainer New CoreContainer 1534671923
   [junit4]   2> 181630 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_3D49B6BCFBA4BC04-001/control-001]
   [junit4]   2> 181630 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_3D49B6BCFBA4BC04-001/control-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_3D49B6BCFBA4BC04-001/control-001/lib
   [junit4]   2> 181630 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 181653 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 181654 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 181654 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 181654 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:52759/solr
   [junit4]   2> 181654 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 181654 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 181655 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 181668 INFO  (zkCallback-140-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@28d9c453 name:ZooKeeperConnection Watcher:127.0.0.1:52759 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 181670 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 181670 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 181701 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [n:127.0.0.1:45254_cu_i%2Ffg    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 181719 INFO  (zkCallback-141-thread-1-processing-n:127.0.0.1:45254_cu_i%2Ffg) [n:127.0.0.1:45254_cu_i%2Ffg    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@26aa762f name:ZooKeeperConnection Watcher:127.0.0.1:52759/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 181719 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [n:127.0.0.1:45254_cu_i%2Ffg    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 181721 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [n:127.0.0.1:45254_cu_i%2Ffg    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 181723 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [n:127.0.0.1:45254_cu_i%2Ffg    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 181725 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [n:127.0.0.1:45254_cu_i%2Ffg    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 181727 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [n:127.0.0.1:45254_cu_i%2Ffg    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 181729 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [n:127.0.0.1:45254_cu_i%2Ffg    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 181731 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [n:127.0.0.1:45254_cu_i%2Ffg    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 181732 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [n:127.0.0.1:45254_cu_i%2Ffg    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 181734 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [n:127.0.0.1:45254_cu_i%2Ffg    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 181735 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [n:127.0.0.1:45254_cu_i%2Ffg    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 181736 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [n:127.0.0.1:45254_cu_i%2Ffg    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:45254_cu_i%2Ffg
   [junit4]   2> 181736 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [n:127.0.0.1:45254_cu_i%2Ffg    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:45254_cu_i%2Ffg
   [junit4]   2> 181737 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [n:127.0.0.1:45254_cu_i%2Ffg    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 181739 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [n:127.0.0.1:45254_cu_i%2Ffg    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 181740 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [n:127.0.0.1:45254_cu_i%2Ffg    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 181741 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [n:127.0.0.1:45254_cu_i%2Ffg    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/95401488593059846-127.0.0.1:45254_cu_i%2Ffg-n_0000000000
   [junit4]   2> 181741 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [n:127.0.0.1:45254_cu_i%2Ffg    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:45254_cu_i%2Ffg
   [junit4]   2> 181741 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [n:127.0.0.1:45254_cu_i%2Ffg    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 181742 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [n:127.0.0.1:45254_cu_i%2Ffg    ] o.a.s.c.Overseer Overseer (id=95401488593059846-127.0.0.1:45254_cu_i%2Ffg-n_0000000000) starting
   [junit4]   2> 181744 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [n:127.0.0.1:45254_cu_i%2Ffg    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 181752 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [n:127.0.0.1:45254_cu_i%2Ffg    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 181758 INFO  (OverseerCollectionConfigSetProcessor-95401488593059846-127.0.0.1:45254_cu_i%2Ffg-n_0000000000) [n:127.0.0.1:45254_cu_i%2Ffg    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 181758 INFO  (OverseerStateUpdate-95401488593059846-127.0.0.1:45254_cu_i%2Ffg-n_0000000000) [n:127.0.0.1:45254_cu_i%2Ffg    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 181759 INFO  (OverseerStateUpdate-95401488593059846-127.0.0.1:45254_cu_i%2Ffg-n_0000000000) [n:127.0.0.1:45254_cu_i%2Ffg    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 181770 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [n:127.0.0.1:45254_cu_i%2Ffg    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 181815 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [n:127.0.0.1:45254_cu_i%2Ffg    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 181815 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [n:127.0.0.1:45254_cu_i%2Ffg    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 181817 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [n:127.0.0.1:45254_cu_i%2Ffg    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_3D49B6BCFBA4BC04-001/control-001/cores
   [junit4]   2> 181819 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [n:127.0.0.1:45254_cu_i%2Ffg    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {shard=, schema=schema.xml, name=collection1, dataDir=data/, loadOnStartup=true, transient=false, collection=control_collection, config=solrconfig.xml, configSetProperties=configsetprops.json, coreNodeName=}
   [junit4]   2> 181820 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [n:127.0.0.1:45254_cu_i%2Ffg    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_3D49B6BCFBA4BC04-001/control-001/cores/collection1
   [junit4]   2> 181820 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [n:127.0.0.1:45254_cu_i%2Ffg    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 181821 INFO  (coreLoadExecutor-386-thread-1-processing-n:127.0.0.1:45254_cu_i%2Ffg) [n:127.0.0.1:45254_cu_i%2Ffg c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 181821 INFO  (coreLoadExecutor-386-thread-1-processing-n:127.0.0.1:45254_cu_i%2Ffg) [n:127.0.0.1:45254_cu_i%2Ffg c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 181836 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [n:127.0.0.1:45254_cu_i%2Ffg    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 181837 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [n:127.0.0.1:45254_cu_i%2Ffg    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 181837 INFO  (OverseerStateUpdate-95401488593059846-127.0.0.1:45254_cu_i%2Ffg-n_0000000000) [n:127.0.0.1:45254_cu_i%2Ffg    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "shard":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:45254/cu_i/fg",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "node_name":"127.0.0.1:45254_cu_i%2Ffg",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "state":"down",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "core":"collection1"} current state version: 0
   [junit4]   2> 181837 INFO  (coreLoadExecutor-386-thread-1-processing-n:127.0.0.1:45254_cu_i%2Ffg) [n:127.0.0.1:45254_cu_i%2Ffg c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 181838 INFO  (OverseerStateUpdate-95401488593059846-127.0.0.1:45254_cu_i%2Ffg-n_0000000000) [n:127.0.0.1:45254_cu_i%2Ffg    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "shard":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:45254/cu_i/fg",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "node_name":"127.0.0.1:45254_cu_i%2Ffg",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "state":"down",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "core":"collection1"}
   [junit4]   2> 181838 INFO  (OverseerStateUpdate-95401488593059846-127.0.0.1:45254_cu_i%2Ffg-n_0000000000) [n:127.0.0.1:45254_cu_i%2Ffg    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 181839 INFO  (OverseerStateUpdate-95401488593059846-127.0.0.1:45254_cu_i%2Ffg-n_0000000000) [n:127.0.0.1:45254_cu_i%2Ffg    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 181842 INFO  (zkCallback-141-thread-1-processing-n:127.0.0.1:45254_cu_i%2Ffg) [n:127.0.0.1:45254_cu_i%2Ffg    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 181848 INFO  (zkCallback-141-thread-2-processing-n:127.0.0.1:45254_cu_i%2Ffg) [n:127.0.0.1:45254_cu_i%2Ffg    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 182838 INFO  (coreLoadExecutor-386-thread-1-processing-n:127.0.0.1:45254_cu_i%2Ffg) [n:127.0.0.1:45254_cu_i%2Ffg c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 182838 INFO  (coreLoadExecutor-386-thread-1-processing-n:127.0.0.1:45254_cu_i%2Ffg) [n:127.0.0.1:45254_cu_i%2Ffg c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 182839 INFO  (coreLoadExecutor-386-thread-1-processing-n:127.0.0.1:45254_cu_i%2Ffg) [n:127.0.0.1:45254_cu_i%2Ffg c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 182839 INFO  (coreLoadExecutor-386-thread-1-processing-n:127.0.0.1:45254_cu_i%2Ffg) [n:127.0.0.1:45254_cu_i%2Ffg c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/control_collection]
   [junit4]   2> 182840 INFO  (coreLoadExecutor-386-thread-1-processing-n:127.0.0.1:45254_cu_i%2Ffg) [n:127.0.0.1:45254_cu_i%2Ffg c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/control_collection] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 182840 INFO  (coreLoadExecutor-386-thread-1-processing-n:127.0.0.1:45254_cu_i%2Ffg) [n:127.0.0.1:45254_cu_i%2Ffg c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_3D49B6BCFBA4BC04-001/control-001/cores/collection1'
   [junit4]   2> 182840 INFO  (coreLoadExecutor-386-thread-1-processing-n:127.0.0.1:45254_cu_i%2Ffg) [n:127.0.0.1:45254_cu_i%2Ffg c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 182840 INFO  (coreLoadExecutor-386-thread-1-processing-n:127.0.0.1:45254_cu_i%2Ffg) [n:127.0.0.1:45254_cu_i%2Ffg c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 182842 INFO  (coreLoadExecutor-386-thread-1-processing-n:127.0.0.1:45254_cu_i%2Ffg) [n:127.0.0.1:45254_cu_i%2Ffg c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 182852 INFO  (coreLoadExecutor-386-thread-1-processing-n:127.0.0.1:45254_cu_i%2Ffg) [n:127.0.0.1:45254_cu_i%2Ffg c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 182866 INFO  (coreLoadExecutor-386-thread-1-processing-n:127.0.0.1:45254_cu_i%2Ffg) [n:127.0.0.1:45254_cu_i%2Ffg c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.6.0
   [junit4]   2> 182898 INFO  (coreLoadExecutor-386-thread-1-processing-n:127.0.0.1:45254_cu_i%2Ffg) [n:127.0.0.1:45254_cu_i%2Ffg c:control_collection   x:collection1] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 182922 INFO  (coreLoadExecutor-386-thread-1-processing-n:127.0.0.1:45254_cu_i%2Ffg) [n:127.0.0.1:45254_cu_i%2Ffg c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 183184 INFO  (coreLoadExecutor-386-thread-1-processing-n:127.0.0.1:45254_cu_i%2Ffg) [n:127.0.0.1:45254_cu_i%2Ffg c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 183198 INFO  (coreLoadExecutor-386-thread-1-processing-n:127.0.0.1:45254_cu_i%2Ffg) [n:127.0.0.1:45254_cu_i%2Ffg c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 183200 INFO  (coreLoadExecutor-386-thread-1-processing-n:127.0.0.1:45254_cu_i%2Ffg) [n:127.0.0.1:45254_cu_i%2Ffg c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 183215 INFO  (coreLoadExecutor-386-thread-1-processing-n:127.0.0.1:45254_cu_i%2Ffg) [n:127.0.0.1:45254_cu_i%2Ffg c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 183220 INFO  (coreLoadExecutor-386-thread-1-processing-n:127.0.0.1:45254_cu_i%2Ffg) [n:127.0.0.1:45254_cu_i%2Ffg c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 183226 INFO  (coreLoadExecutor-386-thread-1-processing-n:127.0.0.1:45254_cu_i%2Ffg) [n:127.0.0.1:45254_cu_i%2Ffg c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 183227 INFO  (coreLoadExecutor-386-thread-1-processing-n:127.0.0.1:45254_cu_i%2Ffg) [n:127.0.0.1:45254_cu_i%2Ffg c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 183232 INFO  (coreLoadExecutor-386-thread-1-processing-n:127.0.0.1:45254_cu_i%2Ffg) [n:127.0.0.1:45254_cu_i%2Ffg c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 183232 INFO  (coreLoadExecutor-386-thread-1-processing-n:127.0.0.1:45254_cu_i%2Ffg) [n:127.0.0.1:45254_cu_i%2Ffg c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 183233 INFO  (coreLoadExecutor-386-thread-1-processing-n:127.0.0.1:45254_cu_i%2Ffg) [n:127.0.0.1:45254_cu_i%2Ffg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 183233 INFO  (coreLoadExecutor-386-thread-1-processing-n:127.0.0.1:45254_cu_i%2Ffg) [n:127.0.0.1:45254_cu_i%2Ffg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_3D49B6BCFBA4BC04-001/control-001/cores/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_3D49B6BCFBA4BC04-001/control-001/cores/collection1/data/]
   [junit4]   2> 183233 INFO  (coreLoadExecutor-386-thread-1-processing-n:127.0.0.1:45254_cu_i%2Ffg) [n:127.0.0.1:45254_cu_i%2Ffg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1347c8ba
   [junit4]   2> 183236 INFO  (coreLoadExecutor-386-thread-1-processing-n:127.0.0.1:45254_cu_i%2Ffg) [n:127.0.0.1:45254_cu_i%2Ffg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_3D49B6BCFBA4BC04-001/control-001/cores/collection1/data
   [junit4]   2> 183237 INFO  (coreLoadExecutor-386-thread-1-processing-n:127.0.0.1:45254_cu_i%2Ffg) [n:127.0.0.1:45254_cu_i%2Ffg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_3D49B6BCFBA4BC04-001/control-001/cores/collection1/data/index/
   [junit4]   2> 183237 WARN  (coreLoadExecutor-386-thread-1-processing-n:127.0.0.1:45254_cu_i%2Ffg) [n:127.0.0.1:45254_cu_i%2Ffg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_3D49B6BCFBA4BC04-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 183238 INFO  (coreLoadExecutor-386-thread-1-processing-n:127.0.0.1:45254_cu_i%2Ffg) [n:127.0.0.1:45254_cu_i%2Ffg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_3D49B6BCFBA4BC04-001/control-001/cores/collection1/data/index
   [junit4]   2> 183238 INFO  (coreLoadExecutor-386-thread-1-processing-n:127.0.0.1:45254_cu_i%2Ffg) [n:127.0.0.1:45254_cu_i%2Ffg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@76bad186
   [junit4]   2> 183238 INFO  (coreLoadExecutor-386-thread-1-processing-n:127.0.0.1:45254_cu_i%2Ffg) [n:127.0.0.1:45254_cu_i%2Ffg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@6311f14f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2ef9ac59),segFN=segments_1,generation=1}
   [junit4]   2> 183239 INFO  (coreLoadExecutor-386-thread-1-processing-n:127.0.0.1:45254_cu_i%2Ffg) [n:127.0.0.1:45254_cu_i%2Ffg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 183239 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:45254_cu_i%2Ffg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_3D49B6BCFBA4BC04-001/control-001/cores/collection1/data/
   [junit4]   2> 183239 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:45254_cu_i%2Ffg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_3D49B6BCFBA4BC04-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 183254 INFO  (coreLoadExecutor-386-thread-1-processing-n:127.0.0.1:45254_cu_i%2Ffg) [n:127.0.0.1:45254_cu_i%2Ffg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 183254 INFO  (coreLoadExecutor-386-thread-1-processing-n:127.0.0.1:45254_cu_i%2Ffg) [n:127.0.0.1:45254_cu_i%2Ffg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 183254 INFO  (coreLoadExecutor-386-thread-1-processing-n:127.0.0.1:45254_cu_i%2Ffg) [n:127.0.0.1:45254_cu_i%2Ffg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 183254 INFO  (coreLoadExecutor-386-thread-1-processing-n:127.0.0.1:45254_cu_i%2Ffg) [n:127.0.0.1:45254_cu_i%2Ffg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 183254 INFO  (coreLoadExecutor-386-thread-1-processing-n:127.0.0.1:45254_cu_i%2Ffg) [n:127.0.0.1:45254_cu_i%2Ffg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 183254 INFO  (coreLoadExecutor-386-thread-1-processing-n:127.0.0.1:45254_cu_i%2Ffg) [n:127.0.0.1:45254_cu_i%2Ffg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 183255 INFO  (coreLoadExecutor-386-thread-1-processing-n:127.0.0.1:45254_cu_i%2Ffg) [n:127.0.0.1:45254_cu_i%2Ffg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 183255 INFO  (coreLoadExecutor-386-thread-1-processing-n:127.0.0.1:45254_cu_i%2Ffg) [n:127.0.0.1:45254_cu_i%2Ffg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 183255 INFO  (coreLoadExecutor-386-thread-1-processing-n:127.0.0.1:45254_cu_i%2Ffg) [n:127.0.0.1:45254_cu_i%2Ffg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 183255 WARN  (coreLoadExecutor-386-thread-1-processing-n:127.0.0.1:45254_cu_i%2Ffg) [n:127.0.0.1:45254_cu_i%2Ffg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
   [junit4]   2> 183258 INFO  (coreLoadExecutor-386-thread-1-processing-n:127.0.0.1:45254_cu_i%2Ffg) [n:127.0.0.1:45254_cu_i%2Ffg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 183260 INFO  (coreLoadExecutor-386-thread-1-processing-n:127.0.0.1:45254_cu_i%2Ffg) [n:127.0.0.1:45254_cu_i%2Ffg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 183261 INFO  (coreLoadExecutor-386-thread-1-processing-n:127.0.0.1:45254_cu_i%2Ffg) [n:127.0.0.1:45254_cu_i%2Ffg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 183263 INFO  (coreLoadExecutor-386-thread-1-processing-n:127.0.0.1:45254_cu_i%2Ffg) [n:127.0.0.1:45254_cu_i%2Ffg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 183268 INFO  (coreLoadExecutor-386-thread-1-processing-n:127.0.0.1:45254_cu_i%2Ffg) [n:127.0.0.1:45254_cu_i%2Ffg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/threads,/admin/luke,/get,/admin/ping,/replication,/admin/system,standard,/admin/properties,/schema,/admin/segments,/admin/plugins,/update,/admin/mbeans,/admin/logging,/update/json,/dump,/update/json/docs,/config,/admin/file,/update/csv
   [junit4]   2> 183269 INFO  (coreLoadExecutor-386-thread-1-processing-n:127.0.0.1:45254_cu_i%2Ffg) [n:127.0.0.1:45254_cu_i%2Ffg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 183270 INFO  (coreLoadExecutor-386-thread-1-processing-n:127.0.0.1:45254_cu_i%2Ffg) [n:127.0.0.1:45254_cu_i%2Ffg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 183270 INFO  (coreLoadExecutor-386-thread-1-processing-n:127.0.0.1:45254_cu_i%2Ffg) [n:127.0.0.1:45254_cu_i%2Ffg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 183271 INFO  (coreLoadExecutor-386-thread-1-processing-n:127.0.0.1:45254_cu_i%2Ffg) [n:127.0.0.1:45254_cu_i%2Ffg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 183271 INFO  (coreLoadExecutor-386-thread-1-processing-n:127.0.0.1:45254_cu_i%2Ffg) [n:127.0.0.1:45254_cu_i%2Ffg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 183271 INFO  (coreLoadExecutor-386-thread-1-processing-n:127.0.0.1:45254_cu_i%2Ffg) [n:127.0.0.1:45254_cu_i%2Ffg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=23, maxMergeAtOnceExplicit=42, maxMergedSegmentMB=89.234375, floorSegmentMB=0.455078125, forceMergeDeletesPctAllowed=12.14149818850256, segmentsPerTier=33.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
   [junit4]   2> 183272 INFO  (coreLoadExecutor-386-thread-1-processing-n:127.0.0.1:45254_cu_i%2Ffg) [n:127.0.0.1:45254_cu_i%2Ffg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@6311f14f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2ef9ac59),segFN=segments_1,generation=1}
   [junit4]   2> 183272 INFO  (coreLoadExecutor-386-thread-1-processing-n:127.0.0.1:45254_cu_i%2Ffg) [n:127.0.0.1:45254_cu_i%2Ffg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 183272 INFO  (coreLoadExecutor-386-thread-1-processing-n:127.0.0.1:45254_cu_i%2Ffg) [n:127.0.0.1:45254_cu_i%2Ffg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@1e724c64[collection1] main
   [junit4]   2> 183273 INFO  (coreLoadExecutor-386-thread-1-processing-n:127.0.0.1:45254_cu_i%2Ffg) [n:127.0.0.1:45254_cu_i%2Ffg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/control_collection]
   [junit4]   2> 183274 INFO  (coreLoadExecutor-386-thread-1-processing-n:127.0.0.1:45254_cu_i%2Ffg) [n:127.0.0.1:45254_cu_i%2Ffg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/control_collection] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 183274 INFO  (coreLoadExecutor-386-thread-1-processing-n:127.0.0.1:45254_cu_i%2Ffg) [n:127.0.0.1:45254_cu_i%2Ffg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 183274 INFO  (coreLoadExecutor-386-thread-1-processing-n:127.0.0.1:45254_cu_i%2Ffg) [n:127.0.0.1:45254_cu_i%2Ffg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 183274 INFO  (coreLoadExecutor-386-thread-1-processing-n:127.0.0.1:45254_cu_i%2Ffg) [n:127.0.0.1:45254_cu_i%2Ffg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 183274 INFO  (coreLoadExecutor-386-thread-1-processing-n:127.0.0.1:45254_cu_i%2Ffg) [n:127.0.0.1:45254_cu_i%2Ffg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 183275 INFO  (coreLoadExecutor-386-thread-1-processing-n:127.0.0.1:45254_cu_i%2Ffg) [n:127.0.0.1:45254_cu_i%2Ffg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 183275 INFO  (coreLoadExecutor-386-thread-1-processing-n:127.0.0.1:45254_cu_i%2Ffg) [n:127.0.0.1:45254_cu_i%2Ffg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 183275 INFO  (coreLoadExecutor-386-thread-1-processing-n:127.0.0.1:45254_cu_i%2Ffg) [n:127.0.0.1:45254_cu_i%2Ffg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 183275 INFO  (coreLoadExecutor-386-thread-1-processing-n:127.0.0.1:45254_cu_i%2Ffg) [n:127.0.0.1:45254_cu_i%2Ffg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 183277 INFO  (searcherExecutor-387-thread-1-processing-s:shard1 r:core_node1 n:127.0.0.1:45254_cu_i%2Ffg c:control_collection x:collection1) [n:127.0.0.1:45254_cu_i%2Ffg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@1e724c64[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 183277 INFO  (coreLoadExecutor-386-thread-1-processing-n:127.0.0.1:45254_cu_i%2Ffg) [n:127.0.0.1:45254_cu_i%2Ffg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 183277 INFO  (coreLoadExecutor-386-thread-1-processing-n:127.0.0.1:45254_cu_i%2Ffg) [n:127.0.0.1:45254_cu_i%2Ffg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 183277 INFO  (coreLoadExecutor-386-thread-1-processing-n:127.0.0.1:45254_cu_i%2Ffg) [n:127.0.0.1:45254_cu_i%2Ffg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 183277 INFO  (coreLoadExecutor-386-thread-1-processing-n:127.0.0.1:45254_cu_i%2Ffg) [n:127.0.0.1:45254_cu_i%2Ffg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1526423819820990464
   [junit4]   2> 183279 INFO  (coreLoadExecutor-386-thread-1-processing-n:127.0.0.1:45254_cu_i%2Ffg) [n:127.0.0.1:45254_cu_i%2Ffg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 2.0ms to seed version buckets with highest version 1526423819820990464
   [junit4]   2> 183280 INFO  (coreLoadExecutor-386-thread-1-processing-n:127.0.0.1:45254_cu_i%2Ffg) [n:127.0.0.1:45254_cu_i%2Ffg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 183280 INFO  (coreLoadExecutor-386-thread-1-processing-n:127.0.0.1:45254_cu_i%2Ffg) [n:127.0.0.1:45254_cu_i%2Ffg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 183281 INFO  (coreZkRegister-380-thread-1-processing-s:shard1 r:core_node1 n:127.0.0.1:45254_cu_i%2Ffg c:control_collection x:collection1) [n:127.0.0.1:45254_cu_i%2Ffg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:45254/cu_i/fg collection:control_collection shard:shard1
   [junit4]   2> 183282 INFO  (coreZkRegister-380-thread-1-processing-s:shard1 r:core_node1 n:127.0.0.1:45254_cu_i%2Ffg c:control_collection x:collection1) [n:127.0.0.1:45254_cu_i%2Ffg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 183285 INFO  (coreZkRegister-380-thread-1-processing-s:shard1 r:core_node1 n:127.0.0.1:45254_cu_i%2Ffg c:control_collection x:collection1) [n:127.0.0.1:45254_cu_i%2Ffg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/95401488593059846-core_node1-n_0000000000
   [junit4]   2> 183286 INFO  (coreZkRegister-380-thread-1-processing-s:shard1 r:core_node1 n:127.0.0.1:45254_cu_i%2Ffg c:control_collection x:collection1) [n:127.0.0.1:45254_cu_i%2Ffg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard=shard1 and weAreReplacement=false and leaderVoteWait=180000
   [junit4]   2> 183287 INFO  (coreZkRegister-380-thread-1-processing-s:shard1 r:core_node1 n:127.0.0.1:45254_cu_i%2Ffg c:control_collection x:collection1) [n:127.0.0.1:45254_cu_i%2Ffg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 183287 INFO  (coreZkRegister-380-thread-1-processing-s:shard1 r:core_node1 n:127.0.0.1:45254_cu_i%2Ffg c:control_collection x:collection1) [n:127.0.0.1:45254_cu_i%2Ffg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 183287 INFO  (coreZkRegister-380-thread-1-processing-s:shard1 r:core_node1 n:127.0.0.1:45254_cu_i%2Ffg c:control_collection x:collection1) [n:127.0.0.1:45254_cu_i%2Ffg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:45254/cu_i/fg/collection1/
   [junit4]   2> 183288 INFO  (coreZkRegister-380-thread-1-processing-s:shard1 r:core_node1 n:127.0.0.1:45254_cu_i%2Ffg c:control_collection x:collection1) [n:127.0.0.1:45254_cu_i%2Ffg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 183288 INFO  (coreZkRegister-380-thread-1-processing-s:shard1 r:core_node1 n:127.0.0.1:45254_cu_i%2Ffg c:control_collection x:collection1) [n:127.0.0.1:45254_cu_i%2Ffg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:45254/cu_i/fg/collection1/ has no replicas
   [junit4]   2> 183288 INFO  (OverseerStateUpdate-95401488593059846-127.0.0.1:45254_cu_i%2Ffg-n_0000000000) [n:127.0.0.1:45254_cu_i%2Ffg    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 183289 INFO  (coreZkRegister-380-thread-1-processing-s:shard1 r:core_node1 n:127.0.0.1:45254_cu_i%2Ffg c:control_collection x:collection1) [n:127.0.0.1:45254_cu_i%2Ffg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 183292 INFO  (coreZkRegister-380-thread-1-processing-s:shard1 r:core_node1 n:127.0.0.1:45254_cu_i%2Ffg c:control_collection x:collection1) [n:127.0.0.1:45254_cu_i%2Ffg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 183295 INFO  (coreZkRegister-380-thread-1-processing-s:shard1 r:core_node1 n:127.0.0.1:45254_cu_i%2Ffg c:control_collection x:collection1) [n:127.0.0.1:45254_cu_i%2Ffg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:45254/cu_i/fg/collection1/ shard1
   [junit4]   2> 183296 INFO  (OverseerStateUpdate-95401488593059846-127.0.0.1:45254_cu_i%2Ffg-n_0000000000) [n:127.0.0.1:45254_cu_i%2Ffg    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:45254/cu_i/fg",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 1
   [junit4]   2> 183339 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 183340 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 183341 INFO  (zkCallback-143-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@26d51159 name:ZooKeeperConnection Watcher:127.0.0.1:52759/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 183342 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 183342 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 183342 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 183347 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 183400 INFO  (zkCallback-143-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 183400 INFO  (zkCallback-141-thread-2-processing-n:127.0.0.1:45254_cu_i%2Ffg) [n:127.0.0.1:45254_cu_i%2Ffg    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 183400 INFO  (zkCallback-141-thread-3-processing-n:127.0.0.1:45254_cu_i%2Ffg) [n:127.0.0.1:45254_cu_i%2Ffg    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 183442 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_3D49B6BCFBA4BC04-001/shard-1-001/cores/collection1
   [junit4]   2> 183443 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_3D49B6BCFBA4BC04-001/shard-1-001
   [junit4]   2> 183444 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 183446 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@76e0fdd0{/cu_i/fg,null,AVAILABLE}
   [junit4]   2> 183446 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.e.j.s.ServerConnector Started ServerConnector@1ed32770{HTTP/1.1}{127.0.0.1:40080}
   [junit4]   2> 183446 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.e.j.s.Server Started @186521ms
   [junit4]   2> 183446 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_3D49B6BCFBA4BC04-001/tempDir-001/jetty1, hostContext=/cu_i/fg, hostPort=40080, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_3D49B6BCFBA4BC04-001/shard-1-001/cores, solrconfig=solrconfig.xml}
   [junit4]   2> 183447 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@41692a49
   [junit4]   2> 183447 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_3D49B6BCFBA4BC04-001/shard-1-001'
   [junit4]   2> 183447 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 183447 INFO  (coreZkRegister-380-thread-1-processing-s:shard1 r:core_node1 n:127.0.0.1:45254_cu_i%2Ffg c:control_collection x:collection1) [n:127.0.0.1:45254_cu_i%2Ffg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:45254/cu_i/fg/collection1/ and leader is http://127.0.0.1:45254/cu_i/fg/collection1/
   [junit4]   2> 183447 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 183447 INFO  (coreZkRegister-380-thread-1-processing-s:shard1 r:core_node1 n:127.0.0.1:45254_cu_i%2Ffg c:control_collection x:collection1) [n:127.0.0.1:45254_cu_i%2Ffg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:45254/cu_i/fg
   [junit4]   2> 183447 INFO  (coreZkRegister-380-thread-1-processing-s:shard1 r:core_node1 n:127.0.0.1:45254_cu_i%2Ffg c:control_collection x:collection1) [n:127.0.0.1:45254_cu_i%2Ffg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 183447 INFO  (coreZkRegister-380-thread-1-processing-s:shard1 r:core_node1 n:127.0.0.1:45254_cu_i%2Ffg c:control_collection x:collection1) [n:127.0.0.1:45254_cu_i%2Ffg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 183447 INFO  (coreZkRegister-380-thread-1-processing-s:shard1 r:core_node1 n:127.0.0.1:45254_cu_i%2Ffg c:control_collection x:collection1) [n:127.0.0.1:45254_cu_i%2Ffg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 183447 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 183448 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 183449 INFO  (zkCallback-144-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2b03330e name:ZooKeeperConnection Watcher:127.0.0.1:52759/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 183450 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 183450 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 183450 INFO  (OverseerStateUpdate-95401488593059846-127.0.0.1:45254_cu_i%2Ffg-n_0000000000) [n:127.0.0.1:45254_cu_i%2Ffg    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "base_url":"http://127.0.0.1:45254/cu_i/fg",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "node_name":"127.0.0.1:45254_cu_i%2Ffg",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "state":"active",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "core":"collection1"} current state version: 2
   [junit4]   2> 183451 INFO  (OverseerStateUpdate-95401488593059846-127.0.0.1:45254_cu_i%2Ffg-n_0000000000) [n:127.0.0.1:45254_cu_i%2Ffg    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "base_url":"http://127.0.0.1:45254/cu_i/fg",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "node_name":"127.0.0.1:45254_cu_i%2Ffg",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "state":"active",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "core":"collection1"}
   [junit4]   2> 183451 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 183451 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_3D49B6BCFBA4BC04-001/shard-1-001/solr.xml
   [junit4]   2> 183475 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_3D49B6BCFBA4BC04-001/shard-1-001/cores
   [junit4]   2> 183475 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.CoreContainer New CoreContainer 38918159
   [junit4]   2> 183475 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_3D49B6BCFBA4BC04-001/shard-1-001]
   [junit4]   2> 183476 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_3D49B6BCFBA4BC04-001/shard-1-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_3D49B6BCFBA4BC04-001/shard-1-001/lib
   [junit4]   2> 183476 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 183477 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 183478 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 183478 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 183478 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:52759/solr
   [junit4]   2> 183478 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 183478 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 183478 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 183480 INFO  (zkCallback-147-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2c5bb455 name:ZooKeeperConnection Watcher:127.0.0.1:52759 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 183480 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 183481 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 183482 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [n:127.0.0.1:40080_cu_i%2Ffg    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 183484 INFO  (zkCallback-148-thread-1-processing-n:127.0.0.1:40080_cu_i%2Ffg) [n:127.0.0.1:40080_cu_i%2Ffg    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3fbd64fe name:ZooKeeperConnection Watcher:127.0.0.1:52759/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 183484 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [n:127.0.0.1:40080_cu_i%2Ffg    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 183490 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[3D49B6BCFBA4BC04]) [n:127.0.0.1:40080_cu_i%2Ffg    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 183554 INFO  (zkCallback-141-thread-3-processing-n:127.0.0.1:45254_cu_i%2Ffg) [n:127.0.0.1:45254_cu_i%2Ffg    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 183554 INFO  (zkCallback-141-thread-2-processing-n:127.0.0.1:45254_cu_i%2Ffg) [n:127.0.0.1:45254_cu_i%2Ffg    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 183554 INFO  (zkCallback-143-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1]

[...truncated too long message...]

SolrCloudCollection_shard1_replica1] o.a.s.c.CachingDirectoryFactory Closing directory: data/index
   [junit4]   2> 11733416 INFO  (jetty-launcher-2599-thread-7) [    ] o.a.s.c.Overseer Overseer (id=95401772438781963-127.0.0.1:44981_solr-n_0000000001) closing
   [junit4]   2> 11733420 INFO  (jetty-launcher-2599-thread-7) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@69afbfa2{/solr,null,UNAVAILABLE}
   [junit4]   2> 11733422 INFO  (jetty-launcher-2599-thread-9) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@368db9ff{/solr,null,UNAVAILABLE}
   [junit4]   2> 11733434 INFO  (coreCloseExecutor-10228-thread-1) [n:127.0.0.1:33428_solr c:testSolrCloudCollection s:shard2 r:core_node4 x:testSolrCloudCollection_shard2_replica2] o.a.s.c.CachingDirectoryFactory Closing RAMDirectoryFactory - 2 directories currently being tracked
   [junit4]   2> 11733435 INFO  (coreCloseExecutor-10228-thread-1) [n:127.0.0.1:33428_solr c:testSolrCloudCollection s:shard2 r:core_node4 x:testSolrCloudCollection_shard2_replica2] o.a.s.c.CachingDirectoryFactory looking to close data/index [CachedDir<<refCount=0;path=data/index;done=false>>]
   [junit4]   2> 11733435 INFO  (coreCloseExecutor-10228-thread-1) [n:127.0.0.1:33428_solr c:testSolrCloudCollection s:shard2 r:core_node4 x:testSolrCloudCollection_shard2_replica2] o.a.s.c.CachingDirectoryFactory Closing directory: data/index
   [junit4]   2> 11733435 INFO  (coreCloseExecutor-10228-thread-1) [n:127.0.0.1:33428_solr c:testSolrCloudCollection s:shard2 r:core_node4 x:testSolrCloudCollection_shard2_replica2] o.a.s.c.CachingDirectoryFactory looking to close data [CachedDir<<refCount=0;path=data;done=false>>]
   [junit4]   2> 11733435 INFO  (coreCloseExecutor-10228-thread-1) [n:127.0.0.1:33428_solr c:testSolrCloudCollection s:shard2 r:core_node4 x:testSolrCloudCollection_shard2_replica2] o.a.s.c.CachingDirectoryFactory Closing directory: data
   [junit4]   2> 11733440 INFO  (jetty-launcher-2599-thread-8) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@1e7c4ee2{/solr,null,UNAVAILABLE}
   [junit4]   2> 11733442 INFO  (TEST-BasicAuthIntegrationTest.testBasics-seed#[3D49B6BCFBA4BC04]) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:44006 44006
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.security.BasicAuthIntegrationTest_3D49B6BCFBA4BC04-001
   [junit4]   2> 11733862 WARN  (jetty-launcher-2599-thread-2-SendThread(127.0.0.1:44006)) [n:127.0.0.1:44981_solr    ] o.a.z.c.StaticHostProvider Unexpected exception
   [junit4]   2> java.lang.InterruptedException: sleep interrupted
   [junit4]   2> 	at java.lang.Thread.sleep(Native Method)
   [junit4]   2> 	at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:940)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1003)
   [junit4]   2> 11733862 WARN  (TEST-BasicAuthIntegrationTest.testBasics-seed#[3D49B6BCFBA4BC04]-SendThread(127.0.0.1:44006)) [n:127.0.0.1:49605_solr    ] o.a.z.c.StaticHostProvider Unexpected exception
   [junit4]   2> java.lang.InterruptedException: sleep interrupted
   [junit4]   2> 	at java.lang.Thread.sleep(Native Method)
   [junit4]   2> 	at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:940)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1003)
   [junit4]   2> Feb 17, 2016 3:25:43 PM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
   [junit4]   2> INFO: All leaked threads terminated.
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene54): {range_facet_l=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), id=Lucene50(blocksize=128), timestamp=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), multiDefault=FSTOrd50, intDefault=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), range_facet_i_dv=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), _version_=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), range_facet_l_dv=Lucene50(blocksize=128)}, docValues:{timestamp=DocValuesFormat(name=Memory), range_facet_i_dv=DocValuesFormat(name=Memory), range_facet_l_dv=DocValuesFormat(name=Lucene54)}, sim=DefaultSimilarity, locale=nl-BE, timezone=America/Santa_Isabel
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_80 (64-bit)/cpus=4,threads=2,free=43864272,total=453509120
   [junit4]   2> NOTE: All tests run in this JVM: [ShardRoutingTest, TestPartialUpdateDeduplication, OverseerStatusTest, TermsComponentTest, TestShardHandlerFactory, SpellingQueryConverterTest, DocValuesTest, BlockCacheTest, ConfigSetsAPITest, TestDistributedGrouping, HdfsDirectoryFactoryTest, CollectionsAPIDistributedZkTest, CollectionTooManyReplicasTest, DirectSolrSpellCheckerTest, TestCloudManagedSchema, TestRequestStatusCollectionAPI, DistributedSuggestComponentTest, ConvertedLegacyTest, RemoteQueryErrorTest, CoreAdminCreateDiscoverTest, TestSurroundQueryParser, TestLFUCache, TestRebalanceLeaders, LoggingHandlerTest, FileUtilsTest, SpellCheckCollatorTest, DistributedIntervalFacetingTest, TestUpdate, TestCustomSort, HdfsBasicDistributedZkTest, TestSolrXml, SoftAutoCommitTest, WordBreakSolrSpellCheckerTest, DOMUtilTest, TestSystemIdResolver, EnumFieldTest, DistanceFunctionTest, DistributedDebugComponentTest, UpdateRequestProcessorFactoryTest, TestBM25SimilarityFactory, HdfsCollectionsAPIDistributedZkTest, ChaosMonkeyNothingIsSafeTest, SolrInfoMBeanTest, TestCollationFieldDocValues, JavabinLoaderTest, SolrPluginUtilsTest, TestDownShardTolerantSearch, SpatialFilterTest, ZkStateWriterTest, DistributedFacetPivotWhiteBoxTest, BlockJoinFacetRandomTest, TestDefaultStatsCache, DistributedTermsComponentTest, MBeansHandlerTest, TestLMJelinekMercerSimilarityFactory, UniqFieldsUpdateProcessorFactoryTest, OpenCloseCoreStressTest, CloudExitableDirectoryReaderTest, OverriddenZkACLAndCredentialsProvidersTest, DistributedFacetPivotLongTailTest, TestConfigSetsAPIZkFailure, CheckHdfsIndexTest, SolrCmdDistributorTest, SparseHLLTest, TestCustomDocTransformer, TestSortByMinMaxFunction, DistributedVersionInfoTest, SchemaVersionSpecificBehaviorTest, BasicFunctionalityTest, TestRemoveLastDynamicCopyField, TestCodecSupport, PreAnalyzedUpdateProcessorTest, SimpleMLTQParserTest, TestSolrCoreProperties, DeleteInactiveReplicaTest, NotRequiredUniqueKeyTest, MoreLikeThisHandlerTest, TestLeaderInitiatedRecoveryThread, TestSimpleTrackingShardHandler, TestSolrCLIRunExample, CSVRequestHandlerTest, TestEmbeddedSolrServerConstructors, ShardSplitTest, TestQueryUtils, ChaosMonkeySafeLeaderTest, DistanceUnitsTest, TestSolrDeletionPolicy2, StressHdfsTest, CachingDirectoryFactoryTest, OpenExchangeRatesOrgProviderTest, NumericFieldsTest, TestSmileRequest, TestCollectionAPI, TestMiniSolrCloudClusterBase, TestMiniSolrCloudClusterSSL, TestReplicaProperties, TriLevelCompositeIdRoutingTest, HdfsChaosMonkeyNothingIsSafeTest, HdfsRecoverLeaseTest, HdfsRecoveryZkTest, HdfsThreadLeakTest, TestClusterStateMutator, RuleEngineTest, RulesTest, TestReplicationHandlerBackup, TestRestoreCore, SegmentsInfoRequestHandlerTest, DistributedFacetPivotSmallAdvancedTest, DistributedFacetPivotSmallTest, FacetPivotSmallTest, SpatialHeatmapFacetsTest, TestDistributedStatsComponentCardinality, TestScoreJoinQPScore, TestNonDefinedSimilarityFactory, TestExactSharedStatsCache, BasicAuthIntegrationTest]
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=BasicAuthIntegrationTest -Dtests.seed=3D49B6BCFBA4BC04 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=nl-BE -Dtests.timezone=America/Santa_Isabel -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
   [junit4] ERROR   0.00s J1 | BasicAuthIntegrationTest (suite) <<<
   [junit4]    > Throwable #1: java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([3D49B6BCFBA4BC04]:0)
   [junit4] Completed [566/566 (3!)] on J1 in 7221.80s, 1 test, 2 errors <<< FAILURES!

[...truncated 14 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:757: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:694: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:534: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1477: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1033: There were test failures: 566 suites (7 ignored), 2279 tests, 2 suite-level errors, 2 errors, 34 ignored (18 assumptions) [seed: 3D49B6BCFBA4BC04]

Total time: 350 minutes 19 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1099 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1099/

3 tests failed.
FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=15528, name=collection4, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=15528, name=collection4, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: java.lang.RuntimeException: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[https://127.0.0.1:53001/tidr/j, https://127.0.0.1:46703/tidr/j, https://127.0.0.1:37737/tidr/j, https://127.0.0.1:54830/tidr/j, https://127.0.0.1:49579/tidr/j]
	at __randomizedtesting.SeedInfo.seed([82ED091DB633FB9F]:0)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:970)
Caused by: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[https://127.0.0.1:53001/tidr/j, https://127.0.0.1:46703/tidr/j, https://127.0.0.1:37737/tidr/j, https://127.0.0.1:54830/tidr/j, https://127.0.0.1:49579/tidr/j]
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:352)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1573)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1594)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:960)
Caused by: org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: https://127.0.0.1:46703/tidr/j
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:585)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:240)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:229)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	... 7 more
Caused by: java.net.SocketTimeoutException: Read timed out
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.read(SocketInputStream.java:152)
	at java.net.SocketInputStream.read(SocketInputStream.java:122)
	at sun.security.ssl.InputRecord.readFully(InputRecord.java:442)
	at sun.security.ssl.InputRecord.read(InputRecord.java:480)
	at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:934)
	at sun.security.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:891)
	at sun.security.ssl.AppInputStream.read(AppInputStream.java:102)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:160)
	at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:84)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:273)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
	at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
	at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)
	at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)
	at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197)
	at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)
	at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)
	at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685)
	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487)
	at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:481)
	... 11 more


FAILED:  org.apache.solr.cloud.DistribDocExpirationUpdateProcessorTest.test

Error Message:
Exactly one shard should have changed, instead: [shard1, shard2] nodes=([core_node4(shard2), core_node1(shard1), core_node3(shard1)]) expected:<1> but was:<2>

Stack Trace:
java.lang.AssertionError: Exactly one shard should have changed, instead: [shard1, shard2] nodes=([core_node4(shard2), core_node1(shard1), core_node3(shard1)]) expected:<1> but was:<2>
	at __randomizedtesting.SeedInfo.seed([82ED091DB633FB9F:AB936C718CF9667]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.failNotEquals(Assert.java:647)
	at org.junit.Assert.assertEquals(Assert.java:128)
	at org.junit.Assert.assertEquals(Assert.java:472)
	at org.apache.solr.cloud.DistribDocExpirationUpdateProcessorTest.test(DistribDocExpirationUpdateProcessorTest.java:118)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:964)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:939)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.handler.TestReplicationHandler.doTestStressReplication

Error Message:
timed out waiting for collection1 startAt time to exceed: Tue Feb 16 08:00:06 ACT 2016

Stack Trace:
java.lang.AssertionError: timed out waiting for collection1 startAt time to exceed: Tue Feb 16 08:00:06 ACT 2016
	at __randomizedtesting.SeedInfo.seed([82ED091DB633FB9F:594609DBB31B922C]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.solr.handler.TestReplicationHandler.watchCoreStartAt(TestReplicationHandler.java:1419)
	at org.apache.solr.handler.TestReplicationHandler.doTestStressReplication(TestReplicationHandler.java:771)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 11575 lines...]
   [junit4] Suite: org.apache.solr.cloud.DistribDocExpirationUpdateProcessorTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.DistribDocExpirationUpdateProcessorTest_82ED091DB633FB9F-001/init-core-data-001
   [junit4]   2> 689553 INFO  (SUITE-DistribDocExpirationUpdateProcessorTest-seed#[82ED091DB633FB9F]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true)
   [junit4]   2> 689553 INFO  (SUITE-DistribDocExpirationUpdateProcessorTest-seed#[82ED091DB633FB9F]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   2> 689560 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 689561 INFO  (Thread-8771) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 689561 INFO  (Thread-8771) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 689661 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [    ] o.a.s.c.ZkTestServer start zk server on port:45489
   [junit4]   2> 689661 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 689670 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 689676 INFO  (zkCallback-905-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@38cc211e name:ZooKeeperConnection Watcher:127.0.0.1:45489 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 689676 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 689677 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 689677 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 689685 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 689686 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 689690 INFO  (zkCallback-906-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@18d68a1c name:ZooKeeperConnection Watcher:127.0.0.1:45489/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 689690 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 689690 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 689690 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 689696 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 689700 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 689702 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 689704 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-doc-expire-update-processor.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 689705 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 689711 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
   [junit4]   2> 689711 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 689716 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 689716 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 689722 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 689722 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 689726 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 689727 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 689730 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 689730 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 689731 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 689732 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 689734 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 689734 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 689735 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 689736 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 689737 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 689737 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 689739 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 689739 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 689844 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.DistribDocExpirationUpdateProcessorTest_82ED091DB633FB9F-001/control-001/cores/collection1
   [junit4]   2> 689846 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 689848 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@3f6588d9{/,null,AVAILABLE}
   [junit4]   2> 689855 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [    ] o.e.j.s.ServerConnector Started ServerConnector@120eb174{HTTP/1.1}{127.0.0.1:44104}
   [junit4]   2> 689855 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [    ] o.e.j.s.Server Started @693161ms
   [junit4]   2> 689855 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.DistribDocExpirationUpdateProcessorTest_82ED091DB633FB9F-001/tempDir-001/control/data, hostContext=/, hostPort=44104, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.DistribDocExpirationUpdateProcessorTest_82ED091DB633FB9F-001/control-001/cores}
   [junit4]   2> 689856 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@3827de24
   [junit4]   2> 689856 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.DistribDocExpirationUpdateProcessorTest_82ED091DB633FB9F-001/control-001'
   [junit4]   2> 689856 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 689856 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 689856 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 689857 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 689858 INFO  (zkCallback-907-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@33d8ca42 name:ZooKeeperConnection Watcher:127.0.0.1:45489/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 689858 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 689859 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 689861 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 689861 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.DistribDocExpirationUpdateProcessorTest_82ED091DB633FB9F-001/control-001/solr.xml
   [junit4]   2> 689876 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.DistribDocExpirationUpdateProcessorTest_82ED091DB633FB9F-001/control-001/cores
   [junit4]   2> 689877 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [    ] o.a.s.c.CoreContainer New CoreContainer 299268765
   [junit4]   2> 689877 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.DistribDocExpirationUpdateProcessorTest_82ED091DB633FB9F-001/control-001]
   [junit4]   2> 689877 WARN  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.DistribDocExpirationUpdateProcessorTest_82ED091DB633FB9F-001/control-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.DistribDocExpirationUpdateProcessorTest_82ED091DB633FB9F-001/control-001/lib
   [junit4]   2> 689877 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 689878 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 689879 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 689879 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 689879 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:45489/solr
   [junit4]   2> 689879 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 689879 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 689880 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 689881 INFO  (zkCallback-910-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@56ff99ed name:ZooKeeperConnection Watcher:127.0.0.1:45489 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 689882 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 689883 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 689886 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [n:127.0.0.1:44104_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 689887 INFO  (zkCallback-911-thread-1-processing-n:127.0.0.1:44104_) [n:127.0.0.1:44104_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@58e291d3 name:ZooKeeperConnection Watcher:127.0.0.1:45489/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 689887 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [n:127.0.0.1:44104_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 689888 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [n:127.0.0.1:44104_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 689893 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [n:127.0.0.1:44104_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 689898 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [n:127.0.0.1:44104_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 689900 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [n:127.0.0.1:44104_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 689901 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [n:127.0.0.1:44104_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 689903 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [n:127.0.0.1:44104_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 689904 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [n:127.0.0.1:44104_    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 689905 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [n:127.0.0.1:44104_    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 689906 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [n:127.0.0.1:44104_    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 689907 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [n:127.0.0.1:44104_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:44104_
   [junit4]   2> 689907 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [n:127.0.0.1:44104_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:44104_
   [junit4]   2> 689908 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [n:127.0.0.1:44104_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 689910 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [n:127.0.0.1:44104_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 689911 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [n:127.0.0.1:44104_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 689912 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [n:127.0.0.1:44104_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/95395920527622148-127.0.0.1:44104_-n_0000000000
   [junit4]   2> 689912 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [n:127.0.0.1:44104_    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:44104_
   [junit4]   2> 689912 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [n:127.0.0.1:44104_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 689913 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [n:127.0.0.1:44104_    ] o.a.s.c.Overseer Overseer (id=95395920527622148-127.0.0.1:44104_-n_0000000000) starting
   [junit4]   2> 689916 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [n:127.0.0.1:44104_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 689923 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [n:127.0.0.1:44104_    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 689923 INFO  (OverseerCollectionConfigSetProcessor-95395920527622148-127.0.0.1:44104_-n_0000000000) [n:127.0.0.1:44104_    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 689923 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [n:127.0.0.1:44104_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 689923 INFO  (OverseerStateUpdate-95395920527622148-127.0.0.1:44104_-n_0000000000) [n:127.0.0.1:44104_    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 689940 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [n:127.0.0.1:44104_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 689940 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [n:127.0.0.1:44104_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 689941 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [n:127.0.0.1:44104_    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.DistribDocExpirationUpdateProcessorTest_82ED091DB633FB9F-001/control-001/cores
   [junit4]   2> 689942 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [n:127.0.0.1:44104_    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {collection=control_collection, dataDir=data/, schema=schema.xml, config=solrconfig.xml, transient=false, loadOnStartup=true, configSetProperties=configsetprops.json, name=collection1, coreNodeName=, shard=}
   [junit4]   2> 689942 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [n:127.0.0.1:44104_    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.DistribDocExpirationUpdateProcessorTest_82ED091DB633FB9F-001/control-001/cores/collection1
   [junit4]   2> 689942 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [n:127.0.0.1:44104_    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 689943 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [n:127.0.0.1:44104_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 689943 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [n:127.0.0.1:44104_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 689944 INFO  (coreLoadExecutor-4968-thread-1-processing-n:127.0.0.1:44104_) [n:127.0.0.1:44104_ c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 689944 INFO  (coreLoadExecutor-4968-thread-1-processing-n:127.0.0.1:44104_) [n:127.0.0.1:44104_ c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 689945 INFO  (OverseerStateUpdate-95395920527622148-127.0.0.1:44104_-n_0000000000) [n:127.0.0.1:44104_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "node_name":"127.0.0.1:44104_",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:44104"} current state version: 0
   [junit4]   2> 689946 INFO  (OverseerStateUpdate-95395920527622148-127.0.0.1:44104_-n_0000000000) [n:127.0.0.1:44104_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "node_name":"127.0.0.1:44104_",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:44104"}
   [junit4]   2> 689946 INFO  (coreLoadExecutor-4968-thread-1-processing-n:127.0.0.1:44104_) [n:127.0.0.1:44104_ c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 689946 INFO  (OverseerStateUpdate-95395920527622148-127.0.0.1:44104_-n_0000000000) [n:127.0.0.1:44104_    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 689947 INFO  (OverseerStateUpdate-95395920527622148-127.0.0.1:44104_-n_0000000000) [n:127.0.0.1:44104_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 689948 INFO  (zkCallback-911-thread-1-processing-n:127.0.0.1:44104_) [n:127.0.0.1:44104_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 690947 INFO  (coreLoadExecutor-4968-thread-1-processing-n:127.0.0.1:44104_) [n:127.0.0.1:44104_ c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 690947 INFO  (coreLoadExecutor-4968-thread-1-processing-n:127.0.0.1:44104_) [n:127.0.0.1:44104_ c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 690948 INFO  (coreLoadExecutor-4968-thread-1-processing-n:127.0.0.1:44104_) [n:127.0.0.1:44104_ c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 690948 INFO  (coreLoadExecutor-4968-thread-1-processing-n:127.0.0.1:44104_) [n:127.0.0.1:44104_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/control_collection]
   [junit4]   2> 690949 INFO  (coreLoadExecutor-4968-thread-1-processing-n:127.0.0.1:44104_) [n:127.0.0.1:44104_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/control_collection] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 690949 INFO  (coreLoadExecutor-4968-thread-1-processing-n:127.0.0.1:44104_) [n:127.0.0.1:44104_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.DistribDocExpirationUpdateProcessorTest_82ED091DB633FB9F-001/control-001/cores/collection1'
   [junit4]   2> 690949 INFO  (coreLoadExecutor-4968-thread-1-processing-n:127.0.0.1:44104_) [n:127.0.0.1:44104_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 690950 INFO  (coreLoadExecutor-4968-thread-1-processing-n:127.0.0.1:44104_) [n:127.0.0.1:44104_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 690951 INFO  (coreLoadExecutor-4968-thread-1-processing-n:127.0.0.1:44104_) [n:127.0.0.1:44104_ c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 690963 INFO  (coreLoadExecutor-4968-thread-1-processing-n:127.0.0.1:44104_) [n:127.0.0.1:44104_ c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 690985 INFO  (coreLoadExecutor-4968-thread-1-processing-n:127.0.0.1:44104_) [n:127.0.0.1:44104_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.6.0
   [junit4]   2> 691039 INFO  (coreLoadExecutor-4968-thread-1-processing-n:127.0.0.1:44104_) [n:127.0.0.1:44104_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 691046 INFO  (coreLoadExecutor-4968-thread-1-processing-n:127.0.0.1:44104_) [n:127.0.0.1:44104_ c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 691283 INFO  (coreLoadExecutor-4968-thread-1-processing-n:127.0.0.1:44104_) [n:127.0.0.1:44104_ c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 691284 INFO  (coreLoadExecutor-4968-thread-1-processing-n:127.0.0.1:44104_) [n:127.0.0.1:44104_ c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 691285 INFO  (coreLoadExecutor-4968-thread-1-processing-n:127.0.0.1:44104_) [n:127.0.0.1:44104_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 691289 INFO  (coreLoadExecutor-4968-thread-1-processing-n:127.0.0.1:44104_) [n:127.0.0.1:44104_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 691337 INFO  (coreLoadExecutor-4968-thread-1-processing-n:127.0.0.1:44104_) [n:127.0.0.1:44104_ c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 691338 INFO  (coreLoadExecutor-4968-thread-1-processing-n:127.0.0.1:44104_) [n:127.0.0.1:44104_ c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 691338 INFO  (coreLoadExecutor-4968-thread-1-processing-n:127.0.0.1:44104_) [n:127.0.0.1:44104_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 691338 INFO  (coreLoadExecutor-4968-thread-1-processing-n:127.0.0.1:44104_) [n:127.0.0.1:44104_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.DistribDocExpirationUpdateProcessorTest_82ED091DB633FB9F-001/control-001/cores/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.DistribDocExpirationUpdateProcessorTest_82ED091DB633FB9F-001/control-001/cores/collection1/data/]
   [junit4]   2> 691338 INFO  (coreLoadExecutor-4968-thread-1-processing-n:127.0.0.1:44104_) [n:127.0.0.1:44104_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore JMX monitoring not detected for core: collection1
   [junit4]   2> 691341 INFO  (coreLoadExecutor-4968-thread-1-processing-n:127.0.0.1:44104_) [n:127.0.0.1:44104_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.DistribDocExpirationUpdateProcessorTest_82ED091DB633FB9F-001/control-001/cores/collection1/data
   [junit4]   2> 691342 INFO  (coreLoadExecutor-4968-thread-1-processing-n:127.0.0.1:44104_) [n:127.0.0.1:44104_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.DistribDocExpirationUpdateProcessorTest_82ED091DB633FB9F-001/control-001/cores/collection1/data/index/
   [junit4]   2> 691342 WARN  (coreLoadExecutor-4968-thread-1-processing-n:127.0.0.1:44104_) [n:127.0.0.1:44104_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.DistribDocExpirationUpdateProcessorTest_82ED091DB633FB9F-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 691342 INFO  (coreLoadExecutor-4968-thread-1-processing-n:127.0.0.1:44104_) [n:127.0.0.1:44104_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.DistribDocExpirationUpdateProcessorTest_82ED091DB633FB9F-001/control-001/cores/collection1/data/index
   [junit4]   2> 691342 INFO  (coreLoadExecutor-4968-thread-1-processing-n:127.0.0.1:44104_) [n:127.0.0.1:44104_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=14, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
   [junit4]   2> 691343 INFO  (coreLoadExecutor-4968-thread-1-processing-n:127.0.0.1:44104_) [n:127.0.0.1:44104_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=RawDirectoryWrapper(RAMDirectory@79dc2163 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@63142915),segFN=segments_1,generation=1}
   [junit4]   2> 691343 INFO  (coreLoadExecutor-4968-thread-1-processing-n:127.0.0.1:44104_) [n:127.0.0.1:44104_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 691353 INFO  (coreLoadExecutor-4968-thread-1-processing-n:127.0.0.1:44104_) [n:127.0.0.1:44104_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "convert-ttl-defaults"
   [junit4]   2> 691355 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:44104_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.DistribDocExpirationUpdateProcessorTest_82ED091DB633FB9F-001/control-001/cores/collection1/data/
   [junit4]   2> 691355 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:44104_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.DistribDocExpirationUpdateProcessorTest_82ED091DB633FB9F-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 691378 INFO  (coreLoadExecutor-4968-thread-1-processing-n:127.0.0.1:44104_) [n:127.0.0.1:44104_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "convert-ttl-field"
   [junit4]   2> 691404 INFO  (coreLoadExecutor-4968-thread-1-processing-n:127.0.0.1:44104_) [n:127.0.0.1:44104_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "convert-ttl-param"
   [junit4]   2> 691404 INFO  (coreLoadExecutor-4968-thread-1-processing-n:127.0.0.1:44104_) [n:127.0.0.1:44104_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "convert-ttl-field-with-param-default"
   [junit4]   2> 691404 INFO  (coreLoadExecutor-4968-thread-1-processing-n:127.0.0.1:44104_) [n:127.0.0.1:44104_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "scheduled-delete" (default)
   [junit4]   2> 691419 INFO  (coreLoadExecutor-4968-thread-1-processing-n:127.0.0.1:44104_) [n:127.0.0.1:44104_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "scheduled-delete" (default)
   [junit4]   2> 691422 INFO  (coreLoadExecutor-4968-thread-1-processing-n:127.0.0.1:44104_) [n:127.0.0.1:44104_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 691424 INFO  (coreLoadExecutor-4968-thread-1-processing-n:127.0.0.1:44104_) [n:127.0.0.1:44104_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 691426 INFO  (coreLoadExecutor-4968-thread-1-processing-n:127.0.0.1:44104_) [n:127.0.0.1:44104_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 691427 INFO  (coreLoadExecutor-4968-thread-1-processing-n:127.0.0.1:44104_) [n:127.0.0.1:44104_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 691431 INFO  (coreLoadExecutor-4968-thread-1-processing-n:127.0.0.1:44104_) [n:127.0.0.1:44104_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /update,/get,/admin/file,/select,/config,/admin/plugins,/admin/mbeans,/admin/system,/admin/logging,/admin/luke,/update/json/docs,/update/json,/schema,/admin/ping,/replication,/admin/threads,,/admin/segments,/admin/properties,/update/csv
   [junit4]   2> 691434 INFO  (coreLoadExecutor-4968-thread-1-processing-n:127.0.0.1:44104_) [n:127.0.0.1:44104_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 691435 INFO  (coreLoadExecutor-4968-thread-1-processing-n:127.0.0.1:44104_) [n:127.0.0.1:44104_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 691435 INFO  (coreLoadExecutor-4968-thread-1-processing-n:127.0.0.1:44104_) [n:127.0.0.1:44104_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 691436 INFO  (coreLoadExecutor-4968-thread-1-processing-n:127.0.0.1:44104_) [n:127.0.0.1:44104_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 691436 INFO  (coreLoadExecutor-4968-thread-1-processing-n:127.0.0.1:44104_) [n:127.0.0.1:44104_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 691436 INFO  (coreLoadExecutor-4968-thread-1-processing-n:127.0.0.1:44104_) [n:127.0.0.1:44104_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=20, maxMergeAtOnceExplicit=21, maxMergedSegmentMB=6.578125, floorSegmentMB=2.0263671875, forceMergeDeletesPctAllowed=26.905184326950796, segmentsPerTier=37.0, maxCFSSegmentSizeMB=1.0068359375, noCFSRatio=0.7377232265961134
   [junit4]   2> 691437 INFO  (coreLoadExecutor-4968-thread-1-processing-n:127.0.0.1:44104_) [n:127.0.0.1:44104_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=RawDirectoryWrapper(RAMDirectory@79dc2163 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@63142915),segFN=segments_1,generation=1}
   [junit4]   2> 691437 INFO  (coreLoadExecutor-4968-thread-1-processing-n:127.0.0.1:44104_) [n:127.0.0.1:44104_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 691437 INFO  (coreLoadExecutor-4968-thread-1-processing-n:127.0.0.1:44104_) [n:127.0.0.1:44104_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@140014eb[collection1] main
   [junit4]   2> 691437 INFO  (coreLoadExecutor-4968-thread-1-processing-n:127.0.0.1:44104_) [n:127.0.0.1:44104_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/control_collection]
   [junit4]   2> 691439 INFO  (coreLoadExecutor-4968-thread-1-processing-n:127.0.0.1:44104_) [n:127.0.0.1:44104_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/control_collection] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 691439 INFO  (coreLoadExecutor-4968-thread-1-processing-n:127.0.0.1:44104_) [n:127.0.0.1:44104_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 691441 INFO  (coreLoadExecutor-4968-thread-1-processing-n:127.0.0.1:44104_) [n:127.0.0.1:44104_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 691441 INFO  (coreLoadExecutor-4968-thread-1-processing-n:127.0.0.1:44104_) [n:127.0.0.1:44104_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 691441 INFO  (coreLoadExecutor-4968-thread-1-processing-n:127.0.0.1:44104_) [n:127.0.0.1:44104_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 691444 INFO  (coreLoadExecutor-4968-thread-1-processing-n:127.0.0.1:44104_) [n:127.0.0.1:44104_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 691444 INFO  (coreLoadExecutor-4968-thread-1-processing-n:127.0.0.1:44104_) [n:127.0.0.1:44104_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 691444 INFO  (coreLoadExecutor-4968-thread-1-processing-n:127.0.0.1:44104_) [n:127.0.0.1:44104_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 691451 INFO  (coreLoadExecutor-4968-thread-1-processing-n:127.0.0.1:44104_) [n:127.0.0.1:44104_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 691451 INFO  (searcherExecutor-4969-thread-1-processing-r:core_node1 x:collection1 n:127.0.0.1:44104_ s:shard1 c:control_collection) [n:127.0.0.1:44104_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@140014eb[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 691451 INFO  (coreLoadExecutor-4968-thread-1-processing-n:127.0.0.1:44104_) [n:127.0.0.1:44104_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 691451 INFO  (coreLoadExecutor-4968-thread-1-processing-n:127.0.0.1:44104_) [n:127.0.0.1:44104_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 691451 INFO  (coreLoadExecutor-4968-thread-1-processing-n:127.0.0.1:44104_) [n:127.0.0.1:44104_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 691451 INFO  (coreLoadExecutor-4968-thread-1-processing-n:127.0.0.1:44104_) [n:127.0.0.1:44104_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1526334730422714368
   [junit4]   2> 691454 INFO  (coreLoadExecutor-4968-thread-1-processing-n:127.0.0.1:44104_) [n:127.0.0.1:44104_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 2.0ms to seed version buckets with highest version 1526334730422714368
   [junit4]   2> 691454 INFO  (coreLoadExecutor-4968-thread-1-processing-n:127.0.0.1:44104_) [n:127.0.0.1:44104_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 691457 INFO  (coreLoadExecutor-4968-thread-1-processing-n:127.0.0.1:44104_) [n:127.0.0.1:44104_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 691459 INFO  (coreZkRegister-4962-thread-1-processing-r:core_node1 x:collection1 n:127.0.0.1:44104_ s:shard1 c:control_collection) [n:127.0.0.1:44104_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:44104 collection:control_collection shard:shard1
   [junit4]   2> 691460 INFO  (coreZkRegister-4962-thread-1-processing-r:core_node1 x:collection1 n:127.0.0.1:44104_ s:shard1 c:control_collection) [n:127.0.0.1:44104_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 691469 INFO  (coreZkRegister-4962-thread-1-processing-r:core_node1 x:collection1 n:127.0.0.1:44104_ s:shard1 c:control_collection) [n:127.0.0.1:44104_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/95395920527622148-core_node1-n_0000000000
   [junit4]   2> 691469 INFO  (coreZkRegister-4962-thread-1-processing-r:core_node1 x:collection1 n:127.0.0.1:44104_ s:shard1 c:control_collection) [n:127.0.0.1:44104_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard=shard1 and weAreReplacement=false and leaderVoteWait=10000
   [junit4]   2> 691471 INFO  (coreZkRegister-4962-thread-1-processing-r:core_node1 x:collection1 n:127.0.0.1:44104_ s:shard1 c:control_collection) [n:127.0.0.1:44104_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 691471 INFO  (coreZkRegister-4962-thread-1-processing-r:core_node1 x:collection1 n:127.0.0.1:44104_ s:shard1 c:control_collection) [n:127.0.0.1:44104_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 691471 INFO  (coreZkRegister-4962-thread-1-processing-r:core_node1 x:collection1 n:127.0.0.1:44104_ s:shard1 c:control_collection) [n:127.0.0.1:44104_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:44104/collection1/
   [junit4]   2> 691471 INFO  (coreZkRegister-4962-thread-1-processing-r:core_node1 x:collection1 n:127.0.0.1:44104_ s:shard1 c:control_collection) [n:127.0.0.1:44104_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 691472 INFO  (OverseerStateUpdate-95395920527622148-127.0.0.1:44104_-n_0000000000) [n:127.0.0.1:44104_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 691472 INFO  (coreZkRegister-4962-thread-1-processing-r:core_node1 x:collection1 n:127.0.0.1:44104_ s:shard1 c:control_collection) [n:127.0.0.1:44104_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:44104/collection1/ has no replicas
   [junit4]   2> 691475 INFO  (coreZkRegister-4962-thread-1-processing-r:core_node1 x:collection1 n:127.0.0.1:44104_ s:shard1 c:control_collection) [n:127.0.0.1:44104_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 691475 INFO  (zkCallback-911-thread-1-processing-n:127.0.0.1:44104_) [n:127.0.0.1:44104_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 691480 INFO  (coreZkRegister-4962-thread-1-processing-r:core_node1 x:collection1 n:127.0.0.1:44104_ s:shard1 c:control_collection) [n:127.0.0.1:44104_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 691486 INFO  (coreZkRegister-4962-thread-1-processing-r:core_node1 x:collection1 n:127.0.0.1:44104_ s:shard1 c:control_collection) [n:127.0.0.1:44104_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:44104/collection1/ shard1
   [junit4]   2> 691486 INFO  (OverseerStateUpdate-95395920527622148-127.0.0.1:44104_-n_0000000000) [n:127.0.0.1:44104_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:44104",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 2
   [junit4]   2> 691590 INFO  (zkCallback-911-thread-2-processing-n:127.0.0.1:44104_) [n:127.0.0.1:44104_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 691638 INFO  (coreZkRegister-4962-thread-1-processing-r:core_node1 x:collection1 n:127.0.0.1:44104_ s:shard1 c:control_collection) [n:127.0.0.1:44104_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:44104/collection1/ and leader is http://127.0.0.1:44104/collection1/
   [junit4]   2> 691638 INFO  (coreZkRegister-4962-thread-1-processing-r:core_node1 x:collection1 n:127.0.0.1:44104_ s:shard1 c:control_collection) [n:127.0.0.1:44104_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:44104
   [junit4]   2> 691638 INFO  (coreZkRegister-4962-thread-1-processing-r:core_node1 x:collection1 n:127.0.0.1:44104_ s:shard1 c:control_collection) [n:127.0.0.1:44104_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 691638 INFO  (coreZkRegister-4962-thread-1-processing-r:core_node1 x:collection1 n:127.0.0.1:44104_ s:shard1 c:control_collection) [n:127.0.0.1:44104_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 691638 INFO  (coreZkRegister-4962-thread-1-processing-r:core_node1 x:collection1 n:127.0.0.1:44104_ s:shard1 c:control_collection) [n:127.0.0.1:44104_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 691640 INFO  (OverseerStateUpdate-95395920527622148-127.0.0.1:44104_-n_0000000000) [n:127.0.0.1:44104_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "node_name":"127.0.0.1:44104_",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:44104"} current state version: 3
   [junit4]   2> 691641 INFO  (OverseerStateUpdate-95395920527622148-127.0.0.1:44104_-n_0000000000) [n:127.0.0.1:44104_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "node_name":"127.0.0.1:44104_",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:44104"}
   [junit4]   2> 691743 INFO  (zkCallback-911-thread-2-processing-n:127.0.0.1:44104_) [n:127.0.0.1:44104_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 691946 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 691946 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 691948 INFO  (zkCallback-913-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3b592dae name:ZooKeeperConnection Watcher:127.0.0.1:45489/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 691948 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 691948 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 691948 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 691950 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 691950 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with stateFormat=2
   [junit4]   2> 691950 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 691951 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 691952 INFO  (zkCallback-914-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3b3bd13 name:ZooKeeperConnection Watcher:127.0.0.1:45489/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 691952 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 691952 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 691954 INFO  (OverseerStateUpdate-95395920527622148-127.0.0.1:44104_-n_0000000000) [n:127.0.0.1:44104_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"create",
   [junit4]   2>   "name":"collection1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "stateFormat":"2"} current state version: 4
   [junit4]   2> 691954 INFO  (OverseerStateUpdate-95395920527622148-127.0.0.1:44104_-n_0000000000) [n:127.0.0.1:44104_    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 692046 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.DistribDocExpirationUpdateProcessorTest_82ED091DB633FB9F-001/shard-1-001/cores/collection1
   [junit4]   2> 692046 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.DistribDocExpirationUpdateProcessorTest_82ED091DB633FB9F-001/shard-1-001
   [junit4]   2> 692047 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 692050 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@40b75e22{/,null,AVAILABLE}
   [junit4]   2> 692050 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [    ] o.e.j.s.ServerConnector Started ServerConnector@50e28e24{HTTP/1.1}{127.0.0.1:39752}
   [junit4]   2> 692050 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [    ] o.e.j.s.Server Started @695356ms
   [junit4]   2> 692050 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.DistribDocExpirationUpdateProcessorTest_82ED091DB633FB9F-001/tempDir-001/jetty1, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.DistribDocExpirationUpdateProcessorTest_82ED091DB633FB9F-001/shard-1-001/cores, hostPort=39752, solrconfig=solrconfig.xml}
   [junit4]   2> 692051 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@3827de24
   [junit4]   2> 692051 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.DistribDocExpirationUpdateProcessorTest_82ED091DB633FB9F-001/shard-1-001'
   [junit4]   2> 692051 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 692051 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 692051 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 692052 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 692056 INFO  (zkCallback-915-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@68ae34c8 name:ZooKeeperConnection Watcher:127.0.0.1:45489/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 692057 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 692057 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 692058 INFO  (OverseerStateUpdate-95395920527622148-127.0.0.1:44104_-n_0000000000) [n:127.0.0.1:44104_    ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/collection1/state.json
   [junit4]   2> 692059 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 692060 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.DistribDocExpirationUpdateProcessorTest_82ED091DB633FB9F-001/shard-1-001/solr.xml
   [junit4]   2> 692076 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.DistribDocExpirationUpdateProcessorTest_82ED091DB633FB9F-001/shard-1-001/cores
   [junit4]   2> 692076 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [    ] o.a.s.c.CoreContainer New CoreContainer 2132096224
   [junit4]   2> 692077 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.DistribDocExpirationUpdateProcessorTest_82ED091DB633FB9F-001/shard-1-001]
   [junit4]   2> 692077 WARN  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.DistribDocExpirationUpdateProcessorTest_82ED091DB633FB9F-001/shard-1-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.DistribDocExpirationUpdateProcessorTest_82ED091DB633FB9F-001/shard-1-001/lib
   [junit4]   2> 692077 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 692078 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 692079 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 692079 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 692079 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:45489/solr
   [junit4]   2> 692079 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 692079 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 692080 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 692082 INFO  (zkCallback-918-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7eea2a6 name:ZooKeeperConnection Watcher:127.0.0.1:45489 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 692084 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 692084 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 692087 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [n:127.0.0.1:39752_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 692089 INFO  (zkCallback-919-thread-1-processing-n:127.0.0.1:39752_) [n:127.0.0.1:39752_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2863c1b5 name:ZooKeeperConnection Watcher:127.0.0.1:45489/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 692089 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [n:127.0.0.1:39752_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 692095 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [n:127.0.0.1:39752_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 693101 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [n:127.0.0.1:39752_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:39752_
   [junit4]   2> 693101 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [n:127.0.0.1:39752_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:39752_
   [junit4]   2> 693102 INFO  (zkCallback-911-thread-2-processing-n:127.0.0.1:44104_) [n:127.0.0.1:44104_    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 693102 INFO  (zkCallback-913-thread-1) [    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 693103 INFO  (zkCallback-919-thread-1-processing-n:127.0.0.1:39752_) [n:127.0.0.1:39752_    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 693106 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [n:127.0.0.1:39752_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 693107 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [n:127.0.0.1:39752_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/95395920527622153-127.0.0.1:39752_-n_0000000001
   [junit4]   2> 693108 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [n:127.0.0.1:39752_    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/95395920527622148-127.0.0.1:44104_-n_0000000000 to know if I could be the leader
   [junit4]   2> 693142 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [n:127.0.0.1:39752_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 693142 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [n:127.0.0.1:39752_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 693143 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [n:127.0.0.1:39752_    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.DistribDocExpirationUpdateProcessorTest_82ED091DB633FB9F-001/shard-1-001/cores
   [junit4]   2> 693144 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [n:127.0.0.1:39752_    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {loadOnStartup=true, transient=false, name=collection1, coreNodeName=, collection=collection1, configSetProperties=configsetprops.json, config=solrconfig.xml, schema=schema.xml, shard=, dataDir=data/}
   [junit4]   2> 693144 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [n:127.0.0.1:39752_    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.DistribDocExpirationUpdateProcessorTest_82ED091DB633FB9F-001/shard-1-001/cores/collection1
   [junit4]   2> 693144 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [n:127.0.0.1:39752_    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 693145 INFO  (coreLoadExecutor-4980-thread-1-processing-n:127.0.0.1:39752_) [n:127.0.0.1:39752_ c:collection1   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 693145 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [n:127.0.0.1:39752_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 693145 INFO  (coreLoadExecutor-4980-thread-1-processing-n:127.0.0.1:39752_) [n:127.0.0.1:39752_ c:collection1   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 693146 INFO  (TEST-DistribDocExpirationUpdateProcessorTest.test-seed#[82ED091DB633FB9F]) [n:127.0.0.1:39752_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 693148 INFO  (OverseerStateUpdate-95395920527622148-127.0.0.1:44104_-n_0000000000) [n:127.0.0.1:44104_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "node_name":"127.0.0.1:39752_",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "base_url":"http://127.0.0.1:39752"} current state version: 4
   [junit4]   2> 693148 INFO  (coreLoadExecutor-4980-thread-1-processing-n:127.0.0.1:39752_) [n:127.0.0.1:39752_ c:collection1   x:collection1] o.a.s.c.ZkController Registering watch for collection collection1
   [junit4]   2> 693148 INFO  (coreLoadExecutor-4980-thread-1-processing-n:127.0.0.1:39752_) [n:127.0.0.1:39752_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader addZkWatch [collection1]
   [junit4]   2> 693149 INFO  (coreLoadExecutor-4980-thread-1-processing-n:127.0.0.1:39752_) [n:127.0.0.1:39752_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Add data for [collection1] ver [0]
   [junit4]   2> 693149 INFO  (coreLoadExecutor-4980-thread-1-processing-n:127.0.0.1:39752_) [n:1

[...truncated too long message...]

ingDirectoryFactory looking to close /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_82ED091DB633FB9F-001/solr-instance-026/./collection1/data [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_82ED091DB633FB9F-001/solr-instance-026/./collection1/data;done=false>>]
   [junit4]   2> 2478120 INFO  (coreCloseExecutor-4001-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_82ED091DB633FB9F-001/solr-instance-026/./collection1/data
   [junit4]   2> 2478120 INFO  (coreCloseExecutor-4001-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory looking to close /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_82ED091DB633FB9F-001/solr-instance-026/./collection1/data/index [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_82ED091DB633FB9F-001/solr-instance-026/./collection1/data/index;done=false>>]
   [junit4]   2> 2478120 INFO  (coreCloseExecutor-4001-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_82ED091DB633FB9F-001/solr-instance-026/./collection1/data/index
   [junit4]   2> 2478120 INFO  (TEST-TestReplicationHandler.doTestDetails-seed#[82ED091DB633FB9F]) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@c0f725c{/solr,null,UNAVAILABLE}
   [junit4]   2> 2478122 INFO  (TEST-TestReplicationHandler.doTestDetails-seed#[82ED091DB633FB9F]) [    ] o.e.j.s.ServerConnector Stopped ServerConnector@31e64d33{HTTP/1.1}{127.0.0.1:0}
   [junit4]   2> 2478122 INFO  (TEST-TestReplicationHandler.doTestDetails-seed#[82ED091DB633FB9F]) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1314770529
   [junit4]   2> 2478123 INFO  (coreCloseExecutor-4002-thread-1) [    x:collection1] o.a.s.c.SolrCore [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@4ac0b8a1
   [junit4]   2> 2478123 INFO  (coreCloseExecutor-4002-thread-1) [    x:collection1] o.a.s.u.DirectUpdateHandler2 closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
   [junit4]   2> 2478123 INFO  (coreCloseExecutor-4002-thread-1) [    x:collection1] o.a.s.u.SolrCoreState Closing SolrCoreState
   [junit4]   2> 2478123 INFO  (coreCloseExecutor-4002-thread-1) [    x:collection1] o.a.s.u.DefaultSolrCoreState SolrCoreState ref count has reached 0 - closing IndexWriter
   [junit4]   2> 2478123 INFO  (coreCloseExecutor-4002-thread-1) [    x:collection1] o.a.s.u.DefaultSolrCoreState closing IndexWriter with IndexWriterCloser
   [junit4]   2> 2478124 INFO  (coreCloseExecutor-4002-thread-1) [    x:collection1] o.a.s.c.SolrCore [collection1] Closing main searcher on request.
   [junit4]   2> 2478124 INFO  (coreCloseExecutor-4002-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing MockDirectoryFactory - 2 directories currently being tracked
   [junit4]   2> 2478124 INFO  (coreCloseExecutor-4002-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory looking to close /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_82ED091DB633FB9F-001/solr-instance-027/./collection1/data/index [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_82ED091DB633FB9F-001/solr-instance-027/./collection1/data/index;done=false>>]
   [junit4]   2> 2478124 INFO  (coreCloseExecutor-4002-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_82ED091DB633FB9F-001/solr-instance-027/./collection1/data/index
   [junit4]   2> 2478124 INFO  (coreCloseExecutor-4002-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory looking to close /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_82ED091DB633FB9F-001/solr-instance-027/./collection1/data [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_82ED091DB633FB9F-001/solr-instance-027/./collection1/data;done=false>>]
   [junit4]   2> 2478124 INFO  (coreCloseExecutor-4002-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_82ED091DB633FB9F-001/solr-instance-027/./collection1/data
   [junit4]   2> 2478124 INFO  (TEST-TestReplicationHandler.doTestDetails-seed#[82ED091DB633FB9F]) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@4eccd5af{/solr,null,UNAVAILABLE}
   [junit4]   2> 2478126 INFO  (SUITE-TestReplicationHandler-seed#[82ED091DB633FB9F]-worker) [    ] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_82ED091DB633FB9F-001
   [junit4]   2> NOTE: test params are: codec=Lucene54, sim=DefaultSimilarity, locale=sl, timezone=America/Eirunepe
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_80 (64-bit)/cpus=4,threads=1,free=281674584,total=522190848
   [junit4]   2> NOTE: All tests run in this JVM: [AsyncCallRequestStatusResponseTest, DocumentAnalysisRequestHandlerTest, TestRawTransformer, TestDocBasedVersionConstraints, TestConfigSets, TestBinaryField, TestScoreJoinQPNoScore, TestOmitPositions, JSONWriterTest, TestRTimerTree, TestHighlightDedupGrouping, TestUniqueKeyFieldResource, TestDefaultSearchFieldResource, TestBulkSchemaAPI, OverseerRolesTest, TestLuceneMatchVersion, ResourceLoaderTest, ResponseHeaderTest, TestConfig, BigEndianAscendingWordDeserializerTest, QueryElevationComponentTest, SaslZkACLProviderTest, UUIDFieldTest, ActionThrottleTest, IndexSchemaRuntimeFieldTest, TestStandardQParsers, SyncSliceTest, VersionInfoTest, TestStressLucene, SegmentsInfoRequestHandlerTest, ZkStateReaderTest, TestReqParamsAPI, PrimUtilsTest, CurrencyFieldOpenExchangeTest, PathHierarchyTokenizerFactoryTest, CursorPagingTest, FileBasedSpellCheckerTest, CleanupOldIndexTest, AddBlockUpdateTest, AlternateDirectoryTest, DistributedQueueTest, BadCopyFieldTest, TestDistribDocBasedVersion, PreAnalyzedFieldTest, TestMissingGroups, TestSolrCloudWithKerberosAlt, DeleteReplicaTest, TestSimpleQParserPlugin, JsonLoaderTest, XsltUpdateRequestHandlerTest, TestLeaderElectionZkExpiry, TestLazyCores, CoreMergeIndexesAdminHandlerTest, SharedFSAutoReplicaFailoverUtilsTest, TestFoldingMultitermQuery, BasicZkTest, TestCrossCoreJoin, TestOrdValues, TestHashQParserPlugin, TestFieldTypeResource, StandardRequestHandlerTest, BlockJoinFacetDistribTest, CopyFieldTest, TestSolrConfigHandlerConcurrent, EchoParamsTest, RecoveryAfterSoftCommitTest, CloudMLTQParserTest, TestBadConfig, HLLUtilTest, TestTrackingShardHandlerFactory, DocValuesMissingTest, OutOfBoxZkACLAndCredentialsProvidersTest, TestAtomicUpdateErrorCases, TestBlendedInfixSuggestions, SynonymTokenizerTest, ParsingFieldUpdateProcessorsTest, TestComponentsName, PolyFieldTest, TestFunctionQuery, TestMiniSolrCloudClusterBase, HdfsChaosMonkeySafeLeaderTest, TestHighFrequencyDictionaryFactory, TestFastOutputStream, BasicDistributedZkTest, TestCloudSchemaless, TestRandomRequestDistribution, SmileWriterTest, TestInfoStreamLogging, ExitableDirectoryReaderTest, HdfsBasicDistributedZk2Test, TestIBSimilarityFactory, DistributedQueryElevationComponentTest, RestartWhileUpdatingTest, OpenCloseCoreStressTest, LeaderElectionIntegrationTest, TestReplicationHandler]
   [junit4] Completed [303/566 (3!)] on J0 in 484.25s, 13 tests, 1 failure <<< FAILURES!

[...truncated 831 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:757: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:694: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:534: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1477: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1033: There were test failures: 566 suites (7 ignored), 2279 tests, 1 error, 2 failures, 34 ignored (18 assumptions) [seed: 82ED091DB633FB9F]

Total time: 207 minutes 26 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



Re: [JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1098 - Still Failing

Posted by Steve Rowe <sa...@gmail.com>.
:) yeah, no coffee yet, thanks Mike.

--
Steve
www.lucidworks.com

> On Feb 15, 2016, at 9:04 AM, Michael McCandless <lu...@mikemccandless.com> wrote:
> 
> I committed a fix for this a few hours ago...
> 
> Mike McCandless
> 
> http://blog.mikemccandless.com
> 
> 
> On Mon, Feb 15, 2016 at 9:01 AM, Steve Rowe <sa...@gmail.com> wrote:
>> I’ve seen this same failure on my Jenkins, but neither seed reproduces for me.
>> 
>> --
>> Steve
>> www.lucidworks.com
>> 
>>> On Feb 15, 2016, at 4:36 AM, Apache Jenkins Server <je...@builds.apache.org> wrote:
>>> 
>>> Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1098/
>>> 
>>> 1 tests failed.
>>> FAILED:  org.apache.lucene.index.TestSortingMergePolicy.testForceMergeNotNeeded
>>> 
>>> Error Message:
>>> 
>>> 
>>> Stack Trace:
>>> java.lang.AssertionError
>>>      at __randomizedtesting.SeedInfo.seed([A5F3779EADC97CBE:B9D00F84369697AB]:0)
>>>      at org.apache.lucene.index.BaseMergePolicyTestCase$1.merge(BaseMergePolicyTestCase.java:43)
>>>      at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:1931)
>>>      at org.apache.lucene.index.IndexWriter.forceMerge(IndexWriter.java:1764)
>>>      at org.apache.lucene.index.IndexWriter.forceMerge(IndexWriter.java:1721)
>>>      at org.apache.lucene.index.BaseMergePolicyTestCase.testForceMergeNotNeeded(BaseMergePolicyTestCase.java:62)
>>>      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>>>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>>      at java.lang.reflect.Method.invoke(Method.java:606)
>>>      at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
>>>      at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
>>>      at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
>>>      at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
>>>      at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
>>>      at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
>>>      at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
>>>      at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
>>>      at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
>>>      at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>>      at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
>>>      at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
>>>      at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
>>>      at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
>>>      at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
>>>      at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
>>>      at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
>>>      at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
>>>      at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>>      at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
>>>      at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
>>>      at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
>>>      at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>>      at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>>      at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>>      at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
>>>      at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
>>>      at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
>>>      at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
>>>      at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>>      at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
>>>      at java.lang.Thread.run(Thread.java:745)
>>> 
>>> 
>>> 
>>> 
>>> Build Log:
>>> [...truncated 8191 lines...]
>>>  [junit4] Suite: org.apache.lucene.index.TestSortingMergePolicy
>>>  [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
>>>  [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestSortingMergePolicy -Dtests.method=testForceMergeNotNeeded -Dtests.seed=A5F3779EADC97CBE -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=ar-DZ -Dtests.timezone=MST7MDT -Dtests.asserts=true -Dtests.file.encoding=UTF-8
>>>  [junit4] FAILURE 3.18s J1 | TestSortingMergePolicy.testForceMergeNotNeeded <<<
>>>  [junit4]    > Throwable #1: java.lang.AssertionError
>>>  [junit4]    >       at __randomizedtesting.SeedInfo.seed([A5F3779EADC97CBE:B9D00F84369697AB]:0)
>>>  [junit4]    >       at org.apache.lucene.index.BaseMergePolicyTestCase$1.merge(BaseMergePolicyTestCase.java:43)
>>>  [junit4]    >       at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:1931)
>>>  [junit4]    >       at org.apache.lucene.index.IndexWriter.forceMerge(IndexWriter.java:1764)
>>>  [junit4]    >       at org.apache.lucene.index.IndexWriter.forceMerge(IndexWriter.java:1721)
>>>  [junit4]    >       at org.apache.lucene.index.BaseMergePolicyTestCase.testForceMergeNotNeeded(BaseMergePolicyTestCase.java:62)
>>>  [junit4]    >       at java.lang.Thread.run(Thread.java:745)
>>>  [junit4]   2> NOTE: test params are: codec=Asserting(Lucene54): {s=FST50}, docValues:{ndv=DocValuesFormat(name=Asserting)}, sim=RandomSimilarity(queryNorm=true,coord=crazy): {}, locale=ar-DZ, timezone=MST7MDT
>>>  [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_80 (64-bit)/cpus=4,threads=1,free=191109240,total=294125568
>>>  [junit4]   2> NOTE: All tests run in this JVM: [TestIndexSplitter, TestFieldCacheReopen, TestHighFreqTerms, IndexSortingTest, TestPKIndexSplitter, TestLazyDocument, TestSortingMergePolicy]
>>>  [junit4] Completed [14/25 (1!)] on J1 in 9.34s, 3 tests, 1 failure <<< FAILURES!
>>> 
>>> [...truncated 48 lines...]
>>> BUILD FAILED
>>> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:757: The following error occurred while executing this line:
>>> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:694: The following error occurred while executing this line:
>>> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
>>> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build.xml:475: The following error occurred while executing this line:
>>> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:2273: The following error occurred while executing this line:
>>> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/module-build.xml:58: The following error occurred while executing this line:
>>> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1477: The following error occurred while executing this line:
>>> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1033: There were test failures: 25 suites, 187 tests, 1 failure, 1 ignored (1 assumption) [seed: A5F3779EADC97CBE]
>>> 
>>> Total time: 105 minutes 12 seconds
>>> Build step 'Invoke Ant' marked build as failure
>>> Archiving artifacts
>>> Recording test results
>>> Email was triggered for: Failure - Any
>>> Sending email for trigger: Failure - Any
>>> 
>>> 
>>> 
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>> For additional commands, e-mail: dev-help@lucene.apache.org
>> 
>> 
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: dev-help@lucene.apache.org
>> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
> 


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Re: [JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1098 - Still Failing

Posted by Michael McCandless <lu...@mikemccandless.com>.
I committed a fix for this a few hours ago...

Mike McCandless

http://blog.mikemccandless.com


On Mon, Feb 15, 2016 at 9:01 AM, Steve Rowe <sa...@gmail.com> wrote:
> I’ve seen this same failure on my Jenkins, but neither seed reproduces for me.
>
> --
> Steve
> www.lucidworks.com
>
>> On Feb 15, 2016, at 4:36 AM, Apache Jenkins Server <je...@builds.apache.org> wrote:
>>
>> Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1098/
>>
>> 1 tests failed.
>> FAILED:  org.apache.lucene.index.TestSortingMergePolicy.testForceMergeNotNeeded
>>
>> Error Message:
>>
>>
>> Stack Trace:
>> java.lang.AssertionError
>>       at __randomizedtesting.SeedInfo.seed([A5F3779EADC97CBE:B9D00F84369697AB]:0)
>>       at org.apache.lucene.index.BaseMergePolicyTestCase$1.merge(BaseMergePolicyTestCase.java:43)
>>       at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:1931)
>>       at org.apache.lucene.index.IndexWriter.forceMerge(IndexWriter.java:1764)
>>       at org.apache.lucene.index.IndexWriter.forceMerge(IndexWriter.java:1721)
>>       at org.apache.lucene.index.BaseMergePolicyTestCase.testForceMergeNotNeeded(BaseMergePolicyTestCase.java:62)
>>       at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>       at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>>       at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>       at java.lang.reflect.Method.invoke(Method.java:606)
>>       at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
>>       at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
>>       at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
>>       at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
>>       at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
>>       at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
>>       at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
>>       at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
>>       at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
>>       at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>       at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
>>       at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
>>       at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
>>       at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
>>       at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
>>       at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
>>       at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
>>       at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
>>       at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>       at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
>>       at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
>>       at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
>>       at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>       at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>       at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>       at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
>>       at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
>>       at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
>>       at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
>>       at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>       at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
>>       at java.lang.Thread.run(Thread.java:745)
>>
>>
>>
>>
>> Build Log:
>> [...truncated 8191 lines...]
>>   [junit4] Suite: org.apache.lucene.index.TestSortingMergePolicy
>>   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
>>   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestSortingMergePolicy -Dtests.method=testForceMergeNotNeeded -Dtests.seed=A5F3779EADC97CBE -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=ar-DZ -Dtests.timezone=MST7MDT -Dtests.asserts=true -Dtests.file.encoding=UTF-8
>>   [junit4] FAILURE 3.18s J1 | TestSortingMergePolicy.testForceMergeNotNeeded <<<
>>   [junit4]    > Throwable #1: java.lang.AssertionError
>>   [junit4]    >       at __randomizedtesting.SeedInfo.seed([A5F3779EADC97CBE:B9D00F84369697AB]:0)
>>   [junit4]    >       at org.apache.lucene.index.BaseMergePolicyTestCase$1.merge(BaseMergePolicyTestCase.java:43)
>>   [junit4]    >       at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:1931)
>>   [junit4]    >       at org.apache.lucene.index.IndexWriter.forceMerge(IndexWriter.java:1764)
>>   [junit4]    >       at org.apache.lucene.index.IndexWriter.forceMerge(IndexWriter.java:1721)
>>   [junit4]    >       at org.apache.lucene.index.BaseMergePolicyTestCase.testForceMergeNotNeeded(BaseMergePolicyTestCase.java:62)
>>   [junit4]    >       at java.lang.Thread.run(Thread.java:745)
>>   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene54): {s=FST50}, docValues:{ndv=DocValuesFormat(name=Asserting)}, sim=RandomSimilarity(queryNorm=true,coord=crazy): {}, locale=ar-DZ, timezone=MST7MDT
>>   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_80 (64-bit)/cpus=4,threads=1,free=191109240,total=294125568
>>   [junit4]   2> NOTE: All tests run in this JVM: [TestIndexSplitter, TestFieldCacheReopen, TestHighFreqTerms, IndexSortingTest, TestPKIndexSplitter, TestLazyDocument, TestSortingMergePolicy]
>>   [junit4] Completed [14/25 (1!)] on J1 in 9.34s, 3 tests, 1 failure <<< FAILURES!
>>
>> [...truncated 48 lines...]
>> BUILD FAILED
>> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:757: The following error occurred while executing this line:
>> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:694: The following error occurred while executing this line:
>> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
>> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build.xml:475: The following error occurred while executing this line:
>> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:2273: The following error occurred while executing this line:
>> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/module-build.xml:58: The following error occurred while executing this line:
>> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1477: The following error occurred while executing this line:
>> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1033: There were test failures: 25 suites, 187 tests, 1 failure, 1 ignored (1 assumption) [seed: A5F3779EADC97CBE]
>>
>> Total time: 105 minutes 12 seconds
>> Build step 'Invoke Ant' marked build as failure
>> Archiving artifacts
>> Recording test results
>> Email was triggered for: Failure - Any
>> Sending email for trigger: Failure - Any
>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: dev-help@lucene.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Re: [JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1098 - Still Failing

Posted by Steve Rowe <sa...@gmail.com>.
I’ve seen this same failure on my Jenkins, but neither seed reproduces for me.

--
Steve
www.lucidworks.com

> On Feb 15, 2016, at 4:36 AM, Apache Jenkins Server <je...@builds.apache.org> wrote:
> 
> Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1098/
> 
> 1 tests failed.
> FAILED:  org.apache.lucene.index.TestSortingMergePolicy.testForceMergeNotNeeded
> 
> Error Message:
> 
> 
> Stack Trace:
> java.lang.AssertionError
> 	at __randomizedtesting.SeedInfo.seed([A5F3779EADC97CBE:B9D00F84369697AB]:0)
> 	at org.apache.lucene.index.BaseMergePolicyTestCase$1.merge(BaseMergePolicyTestCase.java:43)
> 	at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:1931)
> 	at org.apache.lucene.index.IndexWriter.forceMerge(IndexWriter.java:1764)
> 	at org.apache.lucene.index.IndexWriter.forceMerge(IndexWriter.java:1721)
> 	at org.apache.lucene.index.BaseMergePolicyTestCase.testForceMergeNotNeeded(BaseMergePolicyTestCase.java:62)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 	at java.lang.reflect.Method.invoke(Method.java:606)
> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
> 	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
> 	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
> 	at java.lang.Thread.run(Thread.java:745)
> 
> 
> 
> 
> Build Log:
> [...truncated 8191 lines...]
>   [junit4] Suite: org.apache.lucene.index.TestSortingMergePolicy
>   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
>   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestSortingMergePolicy -Dtests.method=testForceMergeNotNeeded -Dtests.seed=A5F3779EADC97CBE -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=ar-DZ -Dtests.timezone=MST7MDT -Dtests.asserts=true -Dtests.file.encoding=UTF-8
>   [junit4] FAILURE 3.18s J1 | TestSortingMergePolicy.testForceMergeNotNeeded <<<
>   [junit4]    > Throwable #1: java.lang.AssertionError
>   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([A5F3779EADC97CBE:B9D00F84369697AB]:0)
>   [junit4]    > 	at org.apache.lucene.index.BaseMergePolicyTestCase$1.merge(BaseMergePolicyTestCase.java:43)
>   [junit4]    > 	at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:1931)
>   [junit4]    > 	at org.apache.lucene.index.IndexWriter.forceMerge(IndexWriter.java:1764)
>   [junit4]    > 	at org.apache.lucene.index.IndexWriter.forceMerge(IndexWriter.java:1721)
>   [junit4]    > 	at org.apache.lucene.index.BaseMergePolicyTestCase.testForceMergeNotNeeded(BaseMergePolicyTestCase.java:62)
>   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
>   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene54): {s=FST50}, docValues:{ndv=DocValuesFormat(name=Asserting)}, sim=RandomSimilarity(queryNorm=true,coord=crazy): {}, locale=ar-DZ, timezone=MST7MDT
>   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_80 (64-bit)/cpus=4,threads=1,free=191109240,total=294125568
>   [junit4]   2> NOTE: All tests run in this JVM: [TestIndexSplitter, TestFieldCacheReopen, TestHighFreqTerms, IndexSortingTest, TestPKIndexSplitter, TestLazyDocument, TestSortingMergePolicy]
>   [junit4] Completed [14/25 (1!)] on J1 in 9.34s, 3 tests, 1 failure <<< FAILURES!
> 
> [...truncated 48 lines...]
> BUILD FAILED
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:757: The following error occurred while executing this line:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:694: The following error occurred while executing this line:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build.xml:475: The following error occurred while executing this line:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:2273: The following error occurred while executing this line:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/module-build.xml:58: The following error occurred while executing this line:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1477: The following error occurred while executing this line:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1033: There were test failures: 25 suites, 187 tests, 1 failure, 1 ignored (1 assumption) [seed: A5F3779EADC97CBE]
> 
> Total time: 105 minutes 12 seconds
> Build step 'Invoke Ant' marked build as failure
> Archiving artifacts
> Recording test results
> Email was triggered for: Failure - Any
> Sending email for trigger: Failure - Any
> 
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1098 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1098/

1 tests failed.
FAILED:  org.apache.lucene.index.TestSortingMergePolicy.testForceMergeNotNeeded

Error Message:


Stack Trace:
java.lang.AssertionError
	at __randomizedtesting.SeedInfo.seed([A5F3779EADC97CBE:B9D00F84369697AB]:0)
	at org.apache.lucene.index.BaseMergePolicyTestCase$1.merge(BaseMergePolicyTestCase.java:43)
	at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:1931)
	at org.apache.lucene.index.IndexWriter.forceMerge(IndexWriter.java:1764)
	at org.apache.lucene.index.IndexWriter.forceMerge(IndexWriter.java:1721)
	at org.apache.lucene.index.BaseMergePolicyTestCase.testForceMergeNotNeeded(BaseMergePolicyTestCase.java:62)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 8191 lines...]
   [junit4] Suite: org.apache.lucene.index.TestSortingMergePolicy
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestSortingMergePolicy -Dtests.method=testForceMergeNotNeeded -Dtests.seed=A5F3779EADC97CBE -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=ar-DZ -Dtests.timezone=MST7MDT -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] FAILURE 3.18s J1 | TestSortingMergePolicy.testForceMergeNotNeeded <<<
   [junit4]    > Throwable #1: java.lang.AssertionError
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([A5F3779EADC97CBE:B9D00F84369697AB]:0)
   [junit4]    > 	at org.apache.lucene.index.BaseMergePolicyTestCase$1.merge(BaseMergePolicyTestCase.java:43)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:1931)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.forceMerge(IndexWriter.java:1764)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.forceMerge(IndexWriter.java:1721)
   [junit4]    > 	at org.apache.lucene.index.BaseMergePolicyTestCase.testForceMergeNotNeeded(BaseMergePolicyTestCase.java:62)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene54): {s=FST50}, docValues:{ndv=DocValuesFormat(name=Asserting)}, sim=RandomSimilarity(queryNorm=true,coord=crazy): {}, locale=ar-DZ, timezone=MST7MDT
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_80 (64-bit)/cpus=4,threads=1,free=191109240,total=294125568
   [junit4]   2> NOTE: All tests run in this JVM: [TestIndexSplitter, TestFieldCacheReopen, TestHighFreqTerms, IndexSortingTest, TestPKIndexSplitter, TestLazyDocument, TestSortingMergePolicy]
   [junit4] Completed [14/25 (1!)] on J1 in 9.34s, 3 tests, 1 failure <<< FAILURES!

[...truncated 48 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:757: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:694: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build.xml:475: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:2273: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/module-build.xml:58: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1477: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1033: There were test failures: 25 suites, 187 tests, 1 failure, 1 ignored (1 assumption) [seed: A5F3779EADC97CBE]

Total time: 105 minutes 12 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1097 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1097/

3 tests failed.
FAILED:  org.apache.solr.hadoop.MorphlineBasicMiniMRTest.mrRun

Error Message:
Failed on local exception: java.io.IOException: Connection reset by peer; Host Details : local host is: "lucene1-us-west/10.41.0.5"; destination host is: "lucene1-us-west.apache.org":49704; 

Stack Trace:
java.io.IOException: Failed on local exception: java.io.IOException: Connection reset by peer; Host Details : local host is: "lucene1-us-west/10.41.0.5"; destination host is: "lucene1-us-west.apache.org":49704; 
	at __randomizedtesting.SeedInfo.seed([F9B45A8DBAB02C2A:F7E6EE83BB261E25]:0)
	at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:772)
	at org.apache.hadoop.ipc.Client.call(Client.java:1472)
	at org.apache.hadoop.ipc.Client.call(Client.java:1399)
	at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:232)
	at com.sun.proxy.$Proxy109.getClusterMetrics(Unknown Source)
	at org.apache.hadoop.yarn.api.impl.pb.client.ApplicationClientProtocolPBClientImpl.getClusterMetrics(ApplicationClientProtocolPBClientImpl.java:202)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:187)
	at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)
	at com.sun.proxy.$Proxy110.getClusterMetrics(Unknown Source)
	at org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.getYarnClusterMetrics(YarnClientImpl.java:461)
	at org.apache.hadoop.mapred.ResourceMgrDelegate.getClusterMetrics(ResourceMgrDelegate.java:151)
	at org.apache.hadoop.mapred.YARNRunner.getClusterMetrics(YARNRunner.java:179)
	at org.apache.hadoop.mapreduce.Cluster.getClusterStatus(Cluster.java:246)
	at org.apache.hadoop.mapred.JobClient$3.run(JobClient.java:719)
	at org.apache.hadoop.mapred.JobClient$3.run(JobClient.java:717)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
	at org.apache.hadoop.mapred.JobClient.getClusterStatus(JobClient.java:717)
	at org.apache.solr.hadoop.MapReduceIndexerTool.run(MapReduceIndexerTool.java:645)
	at org.apache.solr.hadoop.MapReduceIndexerTool.run(MapReduceIndexerTool.java:608)
	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
	at org.apache.solr.hadoop.MorphlineBasicMiniMRTest.mrRun(MorphlineBasicMiniMRTest.java:363)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.IOException: Connection reset by peer
	at sun.nio.ch.FileDispatcherImpl.read0(Native Method)
	at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
	at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223)
	at sun.nio.ch.IOUtil.read(IOUtil.java:197)
	at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:384)
	at org.apache.hadoop.net.SocketInputStream$Reader.performIO(SocketInputStream.java:57)
	at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:142)
	at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161)
	at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131)
	at java.io.FilterInputStream.read(FilterInputStream.java:133)
	at java.io.FilterInputStream.read(FilterInputStream.java:133)
	at org.apache.hadoop.ipc.Client$Connection$PingInputStream.read(Client.java:513)
	at java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
	at java.io.BufferedInputStream.read(BufferedInputStream.java:254)
	at java.io.DataInputStream.readInt(DataInputStream.java:387)
	at org.apache.hadoop.ipc.Client$Connection.receiveRpcResponse(Client.java:1071)
	at org.apache.hadoop.ipc.Client$Connection.run(Client.java:966)


FAILED:  org.apache.solr.hadoop.MorphlineGoLiveMiniMRTest.test

Error Message:
Failed on local exception: java.io.IOException: Connection reset by peer; Host Details : local host is: "lucene1-us-west/10.41.0.5"; destination host is: "lucene1-us-west.apache.org":41858; 

Stack Trace:
java.io.IOException: Failed on local exception: java.io.IOException: Connection reset by peer; Host Details : local host is: "lucene1-us-west/10.41.0.5"; destination host is: "lucene1-us-west.apache.org":41858; 
	at __randomizedtesting.SeedInfo.seed([F9B45A8DBAB02C2A:71E06557144C41D2]:0)
	at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:772)
	at org.apache.hadoop.ipc.Client.call(Client.java:1472)
	at org.apache.hadoop.ipc.Client.call(Client.java:1399)
	at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:232)
	at com.sun.proxy.$Proxy110.getClusterMetrics(Unknown Source)
	at org.apache.hadoop.yarn.api.impl.pb.client.ApplicationClientProtocolPBClientImpl.getClusterMetrics(ApplicationClientProtocolPBClientImpl.java:202)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:187)
	at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)
	at com.sun.proxy.$Proxy111.getClusterMetrics(Unknown Source)
	at org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.getYarnClusterMetrics(YarnClientImpl.java:461)
	at org.apache.hadoop.mapred.ResourceMgrDelegate.getClusterMetrics(ResourceMgrDelegate.java:151)
	at org.apache.hadoop.mapred.YARNRunner.getClusterMetrics(YARNRunner.java:179)
	at org.apache.hadoop.mapreduce.Cluster.getClusterStatus(Cluster.java:246)
	at org.apache.hadoop.mapred.JobClient$3.run(JobClient.java:719)
	at org.apache.hadoop.mapred.JobClient$3.run(JobClient.java:717)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
	at org.apache.hadoop.mapred.JobClient.getClusterStatus(JobClient.java:717)
	at org.apache.solr.hadoop.MapReduceIndexerTool.run(MapReduceIndexerTool.java:645)
	at org.apache.solr.hadoop.MapReduceIndexerTool.run(MapReduceIndexerTool.java:608)
	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
	at org.apache.solr.hadoop.MorphlineGoLiveMiniMRTest.test(MorphlineGoLiveMiniMRTest.java:401)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:964)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:939)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.IOException: Connection reset by peer
	at sun.nio.ch.FileDispatcherImpl.read0(Native Method)
	at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
	at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223)
	at sun.nio.ch.IOUtil.read(IOUtil.java:197)
	at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:384)
	at org.apache.hadoop.net.SocketInputStream$Reader.performIO(SocketInputStream.java:57)
	at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:142)
	at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161)
	at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131)
	at java.io.FilterInputStream.read(FilterInputStream.java:133)
	at java.io.FilterInputStream.read(FilterInputStream.java:133)
	at org.apache.hadoop.ipc.Client$Connection$PingInputStream.read(Client.java:513)
	at java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
	at java.io.BufferedInputStream.read(BufferedInputStream.java:254)
	at java.io.DataInputStream.readInt(DataInputStream.java:387)
	at org.apache.hadoop.ipc.Client$Connection.receiveRpcResponse(Client.java:1071)
	at org.apache.hadoop.ipc.Client$Connection.run(Client.java:966)


FAILED:  junit.framework.TestSuite.org.apache.solr.hadoop.MorphlineGoLiveMiniMRTest

Error Message:
1 thread leaked from SUITE scope at org.apache.solr.hadoop.MorphlineGoLiveMiniMRTest:     1) Thread[id=439, name=Node Status Updater, state=TIMED_WAITING, group=TGRP-MorphlineGoLiveMiniMRTest]         at java.lang.Object.wait(Native Method)         at org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl$1.run(NodeStatusUpdaterImpl.java:659)         at java.lang.Thread.run(Thread.java:745)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.hadoop.MorphlineGoLiveMiniMRTest: 
   1) Thread[id=439, name=Node Status Updater, state=TIMED_WAITING, group=TGRP-MorphlineGoLiveMiniMRTest]
        at java.lang.Object.wait(Native Method)
        at org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl$1.run(NodeStatusUpdaterImpl.java:659)
        at java.lang.Thread.run(Thread.java:745)
	at __randomizedtesting.SeedInfo.seed([F9B45A8DBAB02C2A]:0)




Build Log:
[...truncated 19835 lines...]
   [junit4] Suite: org.apache.solr.hadoop.MorphlineBasicMiniMRTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/init-core-data-001
   [junit4]   2> 0    INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false)
   [junit4]   2> 2401 WARN  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.u.NativeCodeLoader Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
   [junit4]   1> Formatting using clusterid: testClusterID
   [junit4]   2> 2562 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.s.n.FSNamesystem No KeyProvider found.
   [junit4]   2> 2644 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.s.n.FSNamesystem fsLock is fair:true
   [junit4]   2> 2730 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.c.C.deprecation hadoop.configured.node.mapping is deprecated. Instead, use net.topology.configured.node.mapping
   [junit4]   2> 2733 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.s.b.DatanodeManager dfs.block.invalidate.limit=1000
   [junit4]   2> 2734 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.s.b.DatanodeManager dfs.namenode.datanode.registration.ip-hostname-check=true
   [junit4]   2> 2737 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.s.b.BlockManager dfs.namenode.startup.delay.block.deletion.sec is set to 000:00:00:00.000
   [junit4]   2> 2743 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.s.b.BlockManager The block deletion will start around 2016 Shk 14 12:32:02
   [junit4]   2> 2747 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.u.GSet Computing capacity for map BlocksMap
   [junit4]   2> 2748 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.u.GSet VM type       = 64-bit
   [junit4]   2> 2750 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.u.GSet 2.0% max memory 455 MB = 9.1 MB
   [junit4]   2> 2751 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.u.GSet capacity      = 2^20 = 1048576 entries
   [junit4]   2> 2779 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.s.b.BlockManager dfs.block.access.token.enable=false
   [junit4]   2> 2780 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.s.b.BlockManager defaultReplication         = 2
   [junit4]   2> 2780 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.s.b.BlockManager maxReplication             = 512
   [junit4]   2> 2781 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.s.b.BlockManager minReplication             = 1
   [junit4]   2> 2781 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.s.b.BlockManager maxReplicationStreams      = 2
   [junit4]   2> 2781 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.s.b.BlockManager shouldCheckForEnoughRacks  = false
   [junit4]   2> 2782 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.s.b.BlockManager replicationRecheckInterval = 3000
   [junit4]   2> 2782 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.s.b.BlockManager encryptDataTransfer        = false
   [junit4]   2> 2782 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.s.b.BlockManager maxNumBlocksToLog          = 1000
   [junit4]   2> 2792 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.s.n.FSNamesystem fsOwner             = jenkins (auth:SIMPLE)
   [junit4]   2> 2792 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.s.n.FSNamesystem supergroup          = supergroup
   [junit4]   2> 2793 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.s.n.FSNamesystem isPermissionEnabled = true
   [junit4]   2> 2793 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.s.n.FSNamesystem HA Enabled: false
   [junit4]   2> 2799 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.s.n.FSNamesystem Append Enabled: true
   [junit4]   2> 3145 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.u.GSet Computing capacity for map INodeMap
   [junit4]   2> 3146 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.u.GSet VM type       = 64-bit
   [junit4]   2> 3146 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.u.GSet 1.0% max memory 455 MB = 4.5 MB
   [junit4]   2> 3147 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.u.GSet capacity      = 2^19 = 524288 entries
   [junit4]   2> 3147 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.s.n.NameNode Caching file names occuring more than 10 times
   [junit4]   2> 3169 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.u.GSet Computing capacity for map cachedBlocks
   [junit4]   2> 3169 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.u.GSet VM type       = 64-bit
   [junit4]   2> 3169 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.u.GSet 0.25% max memory 455 MB = 1.1 MB
   [junit4]   2> 3170 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.u.GSet capacity      = 2^17 = 131072 entries
   [junit4]   2> 3172 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.s.n.FSNamesystem dfs.namenode.safemode.threshold-pct = 0.9990000128746033
   [junit4]   2> 3173 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.s.n.FSNamesystem dfs.namenode.safemode.min.datanodes = 0
   [junit4]   2> 3173 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.s.n.FSNamesystem dfs.namenode.safemode.extension     = 0
   [junit4]   2> 3176 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.s.n.FSNamesystem Retry cache on namenode is enabled
   [junit4]   2> 3176 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.s.n.FSNamesystem Retry cache will use 0.03 of total heap and retry cache entry expiry time is 600000 millis
   [junit4]   2> 3182 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.u.GSet Computing capacity for map NameNodeRetryCache
   [junit4]   2> 3183 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.u.GSet VM type       = 64-bit
   [junit4]   2> 3183 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.u.GSet 0.029999999329447746% max memory 455 MB = 139.8 KB
   [junit4]   2> 3184 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.u.GSet capacity      = 2^14 = 16384 entries
   [junit4]   2> 3194 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.s.n.NNConf ACLs enabled? false
   [junit4]   2> 3194 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.s.n.NNConf XAttrs enabled? true
   [junit4]   2> 3195 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.s.n.NNConf Maximum size of an xattr: 16384
   [junit4]   2> 3278 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.s.n.FSImage Allocated new BlockPoolId: BP-133164838-10.41.0.5-1455449523010
   [junit4]   2> 3293 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.s.c.Storage Storage directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/name1 has been successfully formatted.
   [junit4]   2> 3299 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.s.c.Storage Storage directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/name2 has been successfully formatted.
   [junit4]   2> 3601 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.s.n.NNStorageRetentionManager Going to retain 1 images with txid >= 0
   [junit4]   2> 3622 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.s.n.NameNode createNameNode []
   [junit4]   2> 3691 WARN  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
   [junit4]   2> 3799 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.m.i.MetricsSystemImpl Scheduled snapshot period at 10 second(s).
   [junit4]   2> 3799 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.m.i.MetricsSystemImpl NameNode metrics system started
   [junit4]   2> 3803 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.s.n.NameNode fs.defaultFS is hdfs://127.0.0.1:0
   [junit4]   2> 3877 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.DFSUtil Starting Web-server for hdfs at: http://localhost:0
   [junit4]   2> 4039 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.m.log Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
   [junit4]   2> 4079 WARN  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 4122 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.HttpServer2 Added global filter 'safety' (class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter)
   [junit4]   2> 4127 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.HttpServer2 Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context hdfs
   [junit4]   2> 4127 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.HttpServer2 Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs
   [junit4]   2> 4127 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.HttpServer2 Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context static
   [junit4]   2> 4181 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.HttpServer2 Added filter 'org.apache.hadoop.hdfs.web.AuthFilter' (class=org.apache.hadoop.hdfs.web.AuthFilter)
   [junit4]   2> 4184 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.HttpServer2 addJerseyResourcePackage: packageName=org.apache.hadoop.hdfs.server.namenode.web.resources;org.apache.hadoop.hdfs.web.resources, pathSpec=/webhdfs/v1/*
   [junit4]   2> 4221 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.HttpServer2 Jetty bound to port 34174
   [junit4]   2> 4221 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 4349 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/hdfs to ./temp/Jetty_localhost_34174_hdfs____ryz793/webapp
   [junit4]   2> 4582 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 5610 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:34174
   [junit4]   2> 5624 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.s.n.FSNamesystem No KeyProvider found.
   [junit4]   2> 5624 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.s.n.FSNamesystem fsLock is fair:true
   [junit4]   2> 5625 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.s.b.DatanodeManager dfs.block.invalidate.limit=1000
   [junit4]   2> 5625 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.s.b.DatanodeManager dfs.namenode.datanode.registration.ip-hostname-check=true
   [junit4]   2> 5626 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.s.b.BlockManager dfs.namenode.startup.delay.block.deletion.sec is set to 000:00:00:00.000
   [junit4]   2> 5626 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.s.b.BlockManager The block deletion will start around 2016 Shk 14 12:32:05
   [junit4]   2> 5626 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.u.GSet Computing capacity for map BlocksMap
   [junit4]   2> 5627 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.u.GSet VM type       = 64-bit
   [junit4]   2> 5627 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.u.GSet 2.0% max memory 455 MB = 9.1 MB
   [junit4]   2> 5627 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.u.GSet capacity      = 2^20 = 1048576 entries
   [junit4]   2> 5628 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.s.b.BlockManager dfs.block.access.token.enable=false
   [junit4]   2> 5628 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.s.b.BlockManager defaultReplication         = 2
   [junit4]   2> 5628 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.s.b.BlockManager maxReplication             = 512
   [junit4]   2> 5629 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.s.b.BlockManager minReplication             = 1
   [junit4]   2> 5629 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.s.b.BlockManager maxReplicationStreams      = 2
   [junit4]   2> 5629 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.s.b.BlockManager shouldCheckForEnoughRacks  = false
   [junit4]   2> 5629 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.s.b.BlockManager replicationRecheckInterval = 3000
   [junit4]   2> 5629 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.s.b.BlockManager encryptDataTransfer        = false
   [junit4]   2> 5630 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.s.b.BlockManager maxNumBlocksToLog          = 1000
   [junit4]   2> 5630 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.s.n.FSNamesystem fsOwner             = jenkins (auth:SIMPLE)
   [junit4]   2> 5630 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.s.n.FSNamesystem supergroup          = supergroup
   [junit4]   2> 5630 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.s.n.FSNamesystem isPermissionEnabled = true
   [junit4]   2> 5631 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.s.n.FSNamesystem HA Enabled: false
   [junit4]   2> 5631 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.s.n.FSNamesystem Append Enabled: true
   [junit4]   2> 5631 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.u.GSet Computing capacity for map INodeMap
   [junit4]   2> 5632 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.u.GSet VM type       = 64-bit
   [junit4]   2> 5632 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.u.GSet 1.0% max memory 455 MB = 4.5 MB
   [junit4]   2> 5632 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.u.GSet capacity      = 2^19 = 524288 entries
   [junit4]   2> 5633 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.s.n.NameNode Caching file names occuring more than 10 times
   [junit4]   2> 5633 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.u.GSet Computing capacity for map cachedBlocks
   [junit4]   2> 5633 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.u.GSet VM type       = 64-bit
   [junit4]   2> 5633 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.u.GSet 0.25% max memory 455 MB = 1.1 MB
   [junit4]   2> 5633 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.u.GSet capacity      = 2^17 = 131072 entries
   [junit4]   2> 5634 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.s.n.FSNamesystem dfs.namenode.safemode.threshold-pct = 0.9990000128746033
   [junit4]   2> 5634 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.s.n.FSNamesystem dfs.namenode.safemode.min.datanodes = 0
   [junit4]   2> 5634 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.s.n.FSNamesystem dfs.namenode.safemode.extension     = 0
   [junit4]   2> 5634 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.s.n.FSNamesystem Retry cache on namenode is enabled
   [junit4]   2> 5635 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.s.n.FSNamesystem Retry cache will use 0.03 of total heap and retry cache entry expiry time is 600000 millis
   [junit4]   2> 5635 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.u.GSet Computing capacity for map NameNodeRetryCache
   [junit4]   2> 5635 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.u.GSet VM type       = 64-bit
   [junit4]   2> 5635 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.u.GSet 0.029999999329447746% max memory 455 MB = 139.8 KB
   [junit4]   2> 5635 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.u.GSet capacity      = 2^14 = 16384 entries
   [junit4]   2> 5638 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.s.n.NNConf ACLs enabled? false
   [junit4]   2> 5638 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.s.n.NNConf XAttrs enabled? true
   [junit4]   2> 5639 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.s.n.NNConf Maximum size of an xattr: 16384
   [junit4]   2> 5653 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.s.c.Storage Lock on /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/name1/in_use.lock acquired by nodename 1055@lucene1-us-west
   [junit4]   2> 5659 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.s.c.Storage Lock on /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/name2/in_use.lock acquired by nodename 1055@lucene1-us-west
   [junit4]   2> 5667 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.s.n.FileJournalManager Recovering unfinalized segments in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/name1/current
   [junit4]   2> 5668 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.s.n.FileJournalManager Recovering unfinalized segments in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/name2/current
   [junit4]   2> 5680 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.s.n.FSImage No edit log streams selected.
   [junit4]   2> 5713 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.s.n.FSImageFormatPBINode Loading 1 INodes.
   [junit4]   2> 5735 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.s.n.FSImageFormatProtobuf Loaded FSImage in 0 seconds.
   [junit4]   2> 5735 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.s.n.FSImage Loaded image for txid 0 from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/name1/current/fsimage_0000000000000000000
   [junit4]   2> 5743 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.s.n.FSNamesystem Need to save fs image? false (staleImage=false, haEnabled=false, isRollingUpgrade=false)
   [junit4]   2> 5743 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.s.n.FSEditLog Starting log segment at 1
   [junit4]   2> 5809 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.s.n.NameCache initialized with 0 entries 0 lookups
   [junit4]   2> 5810 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.s.n.FSNamesystem Finished loading FSImage in 171 msecs
   [junit4]   2> 6230 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.s.n.NameNode RPC server is binding to localhost:0
   [junit4]   2> 6246 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.i.CallQueueManager Using callQueue class java.util.concurrent.LinkedBlockingQueue
   [junit4]   2> 6276 INFO  (Socket Reader #1 for port 41370) [    ] o.a.h.i.Server Starting Socket Reader #1 for port 41370
   [junit4]   2> 6328 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.s.n.NameNode Clients are to use localhost:41370 to access this namenode/service.
   [junit4]   2> 6332 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.s.n.FSNamesystem Registered FSNamesystemState MBean
   [junit4]   2> 6362 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.s.n.FSNamesystem Number of blocks under construction: 0
   [junit4]   2> 6362 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.s.n.FSNamesystem Number of blocks under construction: 0
   [junit4]   2> 6362 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.s.n.FSNamesystem initializing replication queues
   [junit4]   2> 6363 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.StateChange STATE* Leaving safe mode after 0 secs
   [junit4]   2> 6363 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.StateChange STATE* Network topology has 0 racks and 0 datanodes
   [junit4]   2> 6363 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.StateChange STATE* UnderReplicatedBlocks has 0 blocks
   [junit4]   2> 6372 INFO  (Replication Queue Initializer) [    ] o.a.h.h.s.b.BlockManager Total number of blocks            = 0
   [junit4]   2> 6384 INFO  (Replication Queue Initializer) [    ] o.a.h.h.s.b.BlockManager Number of invalid blocks          = 0
   [junit4]   2> 6384 INFO  (Replication Queue Initializer) [    ] o.a.h.h.s.b.BlockManager Number of under-replicated blocks = 0
   [junit4]   2> 6384 INFO  (Replication Queue Initializer) [    ] o.a.h.h.s.b.BlockManager Number of  over-replicated blocks = 0
   [junit4]   2> 6384 INFO  (Replication Queue Initializer) [    ] o.a.h.h.s.b.BlockManager Number of blocks being written    = 0
   [junit4]   2> 6385 INFO  (Replication Queue Initializer) [    ] o.a.h.h.StateChange STATE* Replication Queue initialization scan for invalid, over- and under-replicated blocks completed in 22 msec
   [junit4]   2> 6438 INFO  (IPC Server Responder) [    ] o.a.h.i.Server IPC Server Responder: starting
   [junit4]   2> 6449 INFO  (IPC Server listener on 41370) [    ] o.a.h.i.Server IPC Server listener on 41370: starting
   [junit4]   2> 6450 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.s.n.NameNode NameNode RPC up at: localhost/127.0.0.1:41370
   [junit4]   2> 6450 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.s.n.FSNamesystem Starting services required for active state
   [junit4]   2> 6476 INFO  (CacheReplicationMonitor(1307865526)) [    ] o.a.h.h.s.b.CacheReplicationMonitor Starting CacheReplicationMonitor with interval 30000 milliseconds
   [junit4]   2> 6476 INFO  (CacheReplicationMonitor(1307865526)) [    ] o.a.h.h.s.b.CacheReplicationMonitor Rescanning after 23909815464 milliseconds
   [junit4]   2> 6484 INFO  (CacheReplicationMonitor(1307865526)) [    ] o.a.h.h.s.b.CacheReplicationMonitor Scanned 0 directive(s) and 0 block(s) in 8 millisecond(s).
   [junit4]   2> 6488 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.MiniDFSCluster Starting DataNode 0 with dfs.datanode.data.dir: [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data1,[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data2
   [junit4]   2> 6578 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.m.i.MetricsSystemImpl DataNode metrics system started (again)
   [junit4]   2> 6580 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.s.d.DataNode Configured hostname is 127.0.0.1
   [junit4]   2> 6589 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.s.d.DataNode Starting DataNode with maxLockedMemory = 0
   [junit4]   2> 6599 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.s.d.DataNode Opened streaming server at /127.0.0.1:44115
   [junit4]   2> 6603 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.s.d.DataNode Balancing bandwith is 1048576 bytes/s
   [junit4]   2> 6604 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.s.d.DataNode Number threads for balancing is 5
   [junit4]   2> 6610 WARN  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 6611 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.HttpServer2 Added global filter 'safety' (class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter)
   [junit4]   2> 6612 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.HttpServer2 Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context datanode
   [junit4]   2> 6613 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.HttpServer2 Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context static
   [junit4]   2> 6613 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.HttpServer2 Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs
   [junit4]   2> 6620 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.HttpServer2 addJerseyResourcePackage: packageName=org.apache.hadoop.hdfs.server.datanode.web.resources;org.apache.hadoop.hdfs.web.resources, pathSpec=/webhdfs/v1/*
   [junit4]   2> 6621 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.HttpServer2 Jetty bound to port 57305
   [junit4]   2> 6621 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 6650 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_57305_datanode____qyaf7h/webapp
   [junit4]   2> 6801 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 7355 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:57305
   [junit4]   2> 7358 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.s.d.DataNode dnUserName = jenkins
   [junit4]   2> 7358 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.s.d.DataNode supergroup = supergroup
   [junit4]   2> 7389 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.i.CallQueueManager Using callQueue class java.util.concurrent.LinkedBlockingQueue
   [junit4]   2> 7390 INFO  (Socket Reader #1 for port 41904) [    ] o.a.h.i.Server Starting Socket Reader #1 for port 41904
   [junit4]   2> 7407 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.s.d.DataNode Opened IPC server at /127.0.0.1:41904
   [junit4]   2> 7425 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.s.d.DataNode Refresh request received for nameservices: null
   [junit4]   2> 7429 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.s.d.DataNode Starting BPOfferServices for nameservices: <default>
   [junit4]   2> 7497 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:41370) [    ] o.a.h.h.s.d.DataNode Block pool <registering> (Datanode Uuid unassigned) service to localhost/127.0.0.1:41370 starting to offer service
   [junit4]   2> 7515 INFO  (IPC Server Responder) [    ] o.a.h.i.Server IPC Server Responder: starting
   [junit4]   2> 7533 INFO  (IPC Server listener on 41904) [    ] o.a.h.i.Server IPC Server listener on 41904: starting
   [junit4]   2> 7598 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.MiniDFSCluster Starting DataNode 1 with dfs.datanode.data.dir: [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data3,[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data4
   [junit4]   2> 7689 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.m.i.MetricsSystemImpl DataNode metrics system started (again)
   [junit4]   2> 7689 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.s.d.DataNode Configured hostname is 127.0.0.1
   [junit4]   2> 7689 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.s.d.DataNode Starting DataNode with maxLockedMemory = 0
   [junit4]   2> 7691 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.s.d.DataNode Opened streaming server at /127.0.0.1:41305
   [junit4]   2> 7691 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.s.d.DataNode Balancing bandwith is 1048576 bytes/s
   [junit4]   2> 7691 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.s.d.DataNode Number threads for balancing is 5
   [junit4]   2> 7693 WARN  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 7695 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.HttpServer2 Added global filter 'safety' (class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter)
   [junit4]   2> 7696 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.HttpServer2 Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context datanode
   [junit4]   2> 7696 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.HttpServer2 Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs
   [junit4]   2> 7696 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.HttpServer2 Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context static
   [junit4]   2> 7698 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.HttpServer2 addJerseyResourcePackage: packageName=org.apache.hadoop.hdfs.server.datanode.web.resources;org.apache.hadoop.hdfs.web.resources, pathSpec=/webhdfs/v1/*
   [junit4]   2> 7698 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.HttpServer2 Jetty bound to port 51359
   [junit4]   2> 7699 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 7724 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_51359_datanode____gebkv6/webapp
   [junit4]   2> 7947 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 8462 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:41370) [    ] o.a.h.h.s.c.Storage DataNode version: -56 and NameNode layout version: -60
   [junit4]   2> 8466 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:41370) [    ] o.a.h.h.s.c.Storage Lock on /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data1/in_use.lock acquired by nodename 1055@lucene1-us-west
   [junit4]   2> 8468 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:41370) [    ] o.a.h.h.s.c.Storage Storage directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data1 is not formatted for BP-133164838-10.41.0.5-1455449523010
   [junit4]   2> 8468 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:41370) [    ] o.a.h.h.s.c.Storage Formatting ...
   [junit4]   2> 8475 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:41370) [    ] o.a.h.h.s.c.Storage Lock on /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data2/in_use.lock acquired by nodename 1055@lucene1-us-west
   [junit4]   2> 8476 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:41370) [    ] o.a.h.h.s.c.Storage Storage directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data2 is not formatted for BP-133164838-10.41.0.5-1455449523010
   [junit4]   2> 8477 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:41370) [    ] o.a.h.h.s.c.Storage Formatting ...
   [junit4]   2> 8635 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:51359
   [junit4]   2> 8655 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.s.d.DataNode dnUserName = jenkins
   [junit4]   2> 8655 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.s.d.DataNode supergroup = supergroup
   [junit4]   2> 8656 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.i.CallQueueManager Using callQueue class java.util.concurrent.LinkedBlockingQueue
   [junit4]   2> 8664 INFO  (Socket Reader #1 for port 51143) [    ] o.a.h.i.Server Starting Socket Reader #1 for port 51143
   [junit4]   2> 8672 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.s.d.DataNode Opened IPC server at /127.0.0.1:51143
   [junit4]   2> 8690 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:41370) [    ] o.a.h.h.s.c.Storage Analyzing storage directories for bpid BP-133164838-10.41.0.5-1455449523010
   [junit4]   2> 8691 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:41370) [    ] o.a.h.h.s.c.Storage Locking is disabled
   [junit4]   2> 8693 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:41370) [    ] o.a.h.h.s.c.Storage Storage directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data1/current/BP-133164838-10.41.0.5-1455449523010 is not formatted.
   [junit4]   2> 8693 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:41370) [    ] o.a.h.h.s.c.Storage Formatting ...
   [junit4]   2> 8693 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:41370) [    ] o.a.h.h.s.c.Storage Formatting block pool BP-133164838-10.41.0.5-1455449523010 directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data1/current/BP-133164838-10.41.0.5-1455449523010/current
   [junit4]   2> 8696 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.s.d.DataNode Refresh request received for nameservices: null
   [junit4]   2> 8696 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.h.s.d.DataNode Starting BPOfferServices for nameservices: <default>
   [junit4]   2> 8698 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:41370) [    ] o.a.h.h.s.c.Storage Locking is disabled
   [junit4]   2> 8699 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:41370) [    ] o.a.h.h.s.c.Storage Storage directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data2/current/BP-133164838-10.41.0.5-1455449523010 is not formatted.
   [junit4]   2> 8699 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:41370) [    ] o.a.h.h.s.c.Storage Formatting ...
   [junit4]   2> 8699 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:41370) [    ] o.a.h.h.s.c.Storage Formatting block pool BP-133164838-10.41.0.5-1455449523010 directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data2/current/BP-133164838-10.41.0.5-1455449523010/current
   [junit4]   2> 8703 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:41370) [    ] o.a.h.h.s.c.Storage Restored 0 block files from trash.
   [junit4]   2> 8703 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:41370) [    ] o.a.h.h.s.c.Storage Restored 0 block files from trash.
   [junit4]   2> 8705 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:41370) [    ] o.a.h.h.s.d.DataNode Setting up storage: nsid=349186891;bpid=BP-133164838-10.41.0.5-1455449523010;lv=-56;nsInfo=lv=-60;cid=testClusterID;nsid=349186891;c=0;bpid=BP-133164838-10.41.0.5-1455449523010;dnuuid=null
   [junit4]   2> 8710 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:41370) [    ] o.a.h.h.s.d.DataNode Generated and persisted new Datanode UUID 778c1dbc-6ccd-4a33-b08d-f39cbeffa3b6
   [junit4]   2> 8733 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:41370) [    ] o.a.h.h.s.d.DataNode Block pool <registering> (Datanode Uuid unassigned) service to localhost/127.0.0.1:41370 starting to offer service
   [junit4]   2> 8748 INFO  (IPC Server Responder) [    ] o.a.h.i.Server IPC Server Responder: starting
   [junit4]   2> 8812 INFO  (IPC Server listener on 51143) [    ] o.a.h.i.Server IPC Server listener on 51143: starting
   [junit4]   2> 8841 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:41370) [    ] o.a.h.h.s.c.Storage DataNode version: -56 and NameNode layout version: -60
   [junit4]   2> 8845 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:41370) [    ] o.a.h.h.s.c.Storage Lock on /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data3/in_use.lock acquired by nodename 1055@lucene1-us-west
   [junit4]   2> 8846 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:41370) [    ] o.a.h.h.s.c.Storage Storage directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data3 is not formatted for BP-133164838-10.41.0.5-1455449523010
   [junit4]   2> 8846 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:41370) [    ] o.a.h.h.s.c.Storage Formatting ...
   [junit4]   2> 8851 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:41370) [    ] o.a.h.h.s.c.Storage Lock on /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data4/in_use.lock acquired by nodename 1055@lucene1-us-west
   [junit4]   2> 8852 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:41370) [    ] o.a.h.h.s.c.Storage Storage directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data4 is not formatted for BP-133164838-10.41.0.5-1455449523010
   [junit4]   2> 8852 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:41370) [    ] o.a.h.h.s.c.Storage Formatting ...
   [junit4]   2> 8905 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:41370) [    ] o.a.h.h.s.d.f.i.FsDatasetImpl Added new volume: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data1/current
   [junit4]   2> 8915 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:41370) [    ] o.a.h.h.s.d.f.i.FsDatasetImpl Added volume - /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data1/current, StorageType: DISK
   [junit4]   2> 8916 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:41370) [    ] o.a.h.h.s.d.f.i.FsDatasetImpl Added new volume: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data2/current
   [junit4]   2> 8916 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:41370) [    ] o.a.h.h.s.d.f.i.FsDatasetImpl Added volume - /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data2/current, StorageType: DISK
   [junit4]   2> 8946 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:41370) [    ] o.a.h.h.s.d.f.i.FsDatasetImpl Registered FSDatasetState MBean
   [junit4]   2> 8957 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:41370) [    ] o.a.h.h.s.d.DirectoryScanner Periodic Directory Tree Verification scan starting at 1455449812750 with interval 21600000
   [junit4]   2> 8966 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:41370) [    ] o.a.h.h.s.d.f.i.FsDatasetImpl Adding block pool BP-133164838-10.41.0.5-1455449523010
   [junit4]   2> 8967 INFO  (Thread-90) [    ] o.a.h.h.s.d.f.i.FsDatasetImpl Scanning block pool BP-133164838-10.41.0.5-1455449523010 on volume /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data2/current...
   [junit4]   2> 8974 INFO  (Thread-89) [    ] o.a.h.h.s.d.f.i.FsDatasetImpl Scanning block pool BP-133164838-10.41.0.5-1455449523010 on volume /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data1/current...
   [junit4]   2> 8993 INFO  (Thread-90) [    ] o.a.h.h.s.d.f.i.FsDatasetImpl Time taken to scan block pool BP-133164838-10.41.0.5-1455449523010 on /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data2/current: 22ms
   [junit4]   2> 9013 INFO  (Thread-89) [    ] o.a.h.h.s.d.f.i.FsDatasetImpl Time taken to scan block pool BP-133164838-10.41.0.5-1455449523010 on /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data1/current: 32ms
   [junit4]   2> 9013 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:41370) [    ] o.a.h.h.s.d.f.i.FsDatasetImpl Total time to scan all replicas for block pool BP-133164838-10.41.0.5-1455449523010: 47ms
   [junit4]   2> 9014 INFO  (Thread-94) [    ] o.a.h.h.s.d.f.i.FsDatasetImpl Adding replicas to map for block pool BP-133164838-10.41.0.5-1455449523010 on volume /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data1/current...
   [junit4]   2> 9014 INFO  (Thread-94) [    ] o.a.h.h.s.d.f.i.FsDatasetImpl Time to add replicas to map for block pool BP-133164838-10.41.0.5-1455449523010 on volume /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data1/current: 0ms
   [junit4]   2> 9022 INFO  (Thread-95) [    ] o.a.h.h.s.d.f.i.FsDatasetImpl Adding replicas to map for block pool BP-133164838-10.41.0.5-1455449523010 on volume /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data2/current...
   [junit4]   2> 9024 INFO  (Thread-95) [    ] o.a.h.h.s.d.f.i.FsDatasetImpl Time to add replicas to map for block pool BP-133164838-10.41.0.5-1455449523010 on volume /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data2/current: 0ms
   [junit4]   2> 9024 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:41370) [    ] o.a.h.h.s.d.f.i.FsDatasetImpl Total time to add all replicas to map: 11ms
   [junit4]   2> 9029 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:41370) [    ] o.a.h.h.s.d.DataNode Block pool BP-133164838-10.41.0.5-1455449523010 (Datanode Uuid null) service to localhost/127.0.0.1:41370 beginning handshake with NN
   [junit4]   2> 9036 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:41370) [    ] o.a.h.h.s.c.Storage Analyzing storage directories for bpid BP-133164838-10.41.0.5-1455449523010
   [junit4]   2> 9037 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:41370) [    ] o.a.h.h.s.c.Storage Locking is disabled
   [junit4]   2> 9038 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:41370) [    ] o.a.h.h.s.c.Storage Storage directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data3/current/BP-133164838-10.41.0.5-1455449523010 is not formatted.
   [junit4]   2> 9038 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:41370) [    ] o.a.h.h.s.c.Storage Formatting ...
   [junit4]   2> 9038 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:41370) [    ] o.a.h.h.s.c.Storage Formatting block pool BP-133164838-10.41.0.5-1455449523010 directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data3/current/BP-133164838-10.41.0.5-1455449523010/current
   [junit4]   2> 9042 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:41370) [    ] o.a.h.h.s.c.Storage Locking is disabled
   [junit4]   2> 9043 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:41370) [    ] o.a.h.h.s.c.Storage Storage directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data4/current/BP-133164838-10.41.0.5-1455449523010 is not formatted.
   [junit4]   2> 9043 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBAB02C2A-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:41370) [    ] o.a.h.h.s.c.Storage Formatting ...
   [junit4]   2> 9043 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_F9B45A8DBA

[...truncated too long message...]

oop.metrics2.lib.MetricsSourceBuilder$1.getMetrics(MetricsSourceBuilder.java:79)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.getMetrics(MetricsSourceAdapter.java:195)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.updateJmxCache(MetricsSourceAdapter.java:172)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.getMBeanInfo(MetricsSourceAdapter.java:151)
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getClassName(DefaultMBeanServerInterceptor.java:1804)
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.safeGetClassName(DefaultMBeanServerInterceptor.java:1595)
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.checkMBeanPermission(DefaultMBeanServerInterceptor.java:1813)
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.exclusiveUnregisterMBean(DefaultMBeanServerInterceptor.java:430)
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.unregisterMBean(DefaultMBeanServerInterceptor.java:415)
   [junit4]   2> 	at com.sun.jmx.mbeanserver.JmxMBeanServer.unregisterMBean(JmxMBeanServer.java:546)
   [junit4]   2> 	at org.apache.hadoop.metrics2.util.MBeans.unregister(MBeans.java:81)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.stopMBeans(MetricsSourceAdapter.java:227)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.stop(MetricsSourceAdapter.java:212)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.stopSources(MetricsSystemImpl.java:461)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.stop(MetricsSystemImpl.java:212)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.shutdown(MetricsSystemImpl.java:592)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.shutdownInstance(DefaultMetricsSystem.java:72)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.shutdown(DefaultMetricsSystem.java:68)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.metrics.NameNodeMetrics.shutdown(NameNodeMetrics.java:145)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.NameNode.stop(NameNode.java:822)
   [junit4]   2> 	at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1720)
   [junit4]   2> 	at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1699)
   [junit4]   2> 	at org.apache.solr.hadoop.MorphlineGoLiveMiniMRTest.teardownClass(MorphlineGoLiveMiniMRTest.java:235)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   [junit4]   2> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2> 	at java.lang.reflect.Method.invoke(Method.java:606)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:834)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> Caused by: java.lang.NullPointerException
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.blockmanagement.BlocksMap.size(BlocksMap.java:198)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.getTotalBlocks(BlockManager.java:3291)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlocksTotal(FSNamesystem.java:6223)
   [junit4]   2> 	... 53 more
   [junit4]   2> 82199 INFO  (SUITE-MorphlineGoLiveMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.m.i.MetricsSystemImpl JobHistoryServer metrics system stopped.
   [junit4]   2> 82200 INFO  (SUITE-MorphlineGoLiveMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.h.m.i.MetricsSystemImpl JobHistoryServer metrics system shutdown complete.
   [junit4]   2> 82201 INFO  (SUITE-MorphlineGoLiveMiniMRTest-seed#[F9B45A8DBAB02C2A]-worker) [    ] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J1/temp/solr.hadoop.MorphlineGoLiveMiniMRTest_F9B45A8DBAB02C2A-001
   [junit4]   2> Feb 14, 2016 11:33:22 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> SEVERE: 1 thread leaked from SUITE scope at org.apache.solr.hadoop.MorphlineGoLiveMiniMRTest: 
   [junit4]   2>    1) Thread[id=439, name=Node Status Updater, state=TIMED_WAITING, group=TGRP-MorphlineGoLiveMiniMRTest]
   [junit4]   2>         at java.lang.Object.wait(Native Method)
   [junit4]   2>         at org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl$1.run(NodeStatusUpdaterImpl.java:659)
   [junit4]   2>         at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene54), sim=RandomSimilarity(queryNorm=false,coord=yes): {}, locale=uk-UA, timezone=US/East-Indiana
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_80 (64-bit)/cpus=4,threads=9,free=119738168,total=345505792
   [junit4]   2> NOTE: All tests run in this JVM: [MorphlineGoLiveMiniMRTest]
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=MorphlineGoLiveMiniMRTest -Dtests.seed=F9B45A8DBAB02C2A -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=uk-UA -Dtests.timezone=US/East-Indiana -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
   [junit4] ERROR   0.00s J1 | MorphlineGoLiveMiniMRTest (suite) <<<
   [junit4]    > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.hadoop.MorphlineGoLiveMiniMRTest: 
   [junit4]    >    1) Thread[id=439, name=Node Status Updater, state=TIMED_WAITING, group=TGRP-MorphlineGoLiveMiniMRTest]
   [junit4]    >         at java.lang.Object.wait(Native Method)
   [junit4]    >         at org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl$1.run(NodeStatusUpdaterImpl.java:659)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([F9B45A8DBAB02C2A]:0)
   [junit4] Completed [7/7 (2!)] on J1 in 84.01s, 2 tests, 2 errors <<< FAILURES!

[...truncated 13 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:757: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:694: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:253: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:469: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:534: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1477: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1033: There were test failures: 7 suites, 35 tests, 1 suite-level error, 2 errors, 1 ignored [seed: F9B45A8DBAB02C2A]

Total time: 209 minutes 17 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1096 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1096/

2 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.handler.TestReplicationHandler

Error Message:
ObjectTracker found 2 object(s) that were not released!!! [NRTCachingDirectory, NRTCachingDirectory]

Stack Trace:
java.lang.AssertionError: ObjectTracker found 2 object(s) that were not released!!! [NRTCachingDirectory, NRTCachingDirectory]
	at __randomizedtesting.SeedInfo.seed([A17758EE270828B3]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.junit.Assert.assertNull(Assert.java:551)
	at org.apache.solr.SolrTestCaseJ4.afterClass(SolrTestCaseJ4.java:228)
	at sun.reflect.GeneratedMethodAccessor58.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:834)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.search.TestIndexSearcher.testReopen

Error Message:
nothing changed, searcher should be the same expected same:<Searcher@7af24215[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(5.6.0):c2) Uninverting(_1(5.6.0):c2) Uninverting(_2(5.6.0):c2)))}> was not:<Searcher@1fe5e594[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(5.6.0):c2) Uninverting(_1(5.6.0):c2) Uninverting(_2(5.6.0):c2)))}>

Stack Trace:
java.lang.AssertionError: nothing changed, searcher should be the same expected same:<Searcher@7af24215[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(5.6.0):c2) Uninverting(_1(5.6.0):c2) Uninverting(_2(5.6.0):c2)))}> was not:<Searcher@1fe5e594[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(5.6.0):c2) Uninverting(_1(5.6.0):c2) Uninverting(_2(5.6.0):c2)))}>
	at __randomizedtesting.SeedInfo.seed([A17758EE270828B3:8D3F89F85434A790]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.failNotSame(Assert.java:641)
	at org.junit.Assert.assertSame(Assert.java:580)
	at org.apache.solr.search.TestIndexSearcher.testReopen(TestIndexSearcher.java:134)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 11662 lines...]
   [junit4] JVM J1: stdout was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J1-20160213_092621_820.sysout
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] java.lang.OutOfMemoryError: GC overhead limit exceeded
   [junit4] Dumping heap to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/heapdumps/java_pid16312.hprof ...
   [junit4] Heap dump file created [612537982 bytes in 10.829 secs]
   [junit4] <<< JVM J1: EOF ----

   [junit4] JVM J1: stderr was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J1-20160213_092621_820.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] 
   [junit4] Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0"
   [junit4] WARN: Unhandled exception in event serialization. -> java.lang.OutOfMemoryError: GC overhead limit exceeded
   [junit4] <<< JVM J1: EOF ----

[...truncated 8 lines...]
   [junit4] Suite: org.apache.solr.handler.TestReplicationHandler
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A17758EE270828B3-001/init-core-data-001
   [junit4]   2> 1129771 INFO  (TEST-TestReplicationHandler.doTestStressReplication-seed#[A17758EE270828B3]) [    ] o.a.s.SolrTestCaseJ4 ###Starting doTestStressReplication
   [junit4]   2> 1129772 INFO  (TEST-TestReplicationHandler.doTestStressReplication-seed#[A17758EE270828B3]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A17758EE270828B3-001/solr-instance-001/collection1
   [junit4]   2> 1129775 INFO  (TEST-TestReplicationHandler.doTestStressReplication-seed#[A17758EE270828B3]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 1129777 INFO  (TEST-TestReplicationHandler.doTestStressReplication-seed#[A17758EE270828B3]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@61be759f{/solr,null,AVAILABLE}
   [junit4]   2> 1129777 INFO  (TEST-TestReplicationHandler.doTestStressReplication-seed#[A17758EE270828B3]) [    ] o.e.j.s.ServerConnector Started ServerConnector@7ab826d1{HTTP/1.1}{127.0.0.1:38378}
   [junit4]   2> 1129778 INFO  (TEST-TestReplicationHandler.doTestStressReplication-seed#[A17758EE270828B3]) [    ] o.e.j.s.Server Started @1132928ms
   [junit4]   2> 1129778 INFO  (TEST-TestReplicationHandler.doTestStressReplication-seed#[A17758EE270828B3]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A17758EE270828B3-001/solr-instance-001/collection1/data, hostPort=38378, hostContext=/solr}
   [junit4]   2> 1129778 INFO  (TEST-TestReplicationHandler.doTestStressReplication-seed#[A17758EE270828B3]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@3827de24
   [junit4]   2> 1129778 INFO  (TEST-TestReplicationHandler.doTestStressReplication-seed#[A17758EE270828B3]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A17758EE270828B3-001/solr-instance-001'
   [junit4]   2> 1129778 INFO  (TEST-TestReplicationHandler.doTestStressReplication-seed#[A17758EE270828B3]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1129778 INFO  (TEST-TestReplicationHandler.doTestStressReplication-seed#[A17758EE270828B3]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1129778 INFO  (TEST-TestReplicationHandler.doTestStressReplication-seed#[A17758EE270828B3]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A17758EE270828B3-001/solr-instance-001/solr.xml
   [junit4]   2> 1129792 INFO  (TEST-TestReplicationHandler.doTestStressReplication-seed#[A17758EE270828B3]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A17758EE270828B3-001/solr-instance-001/.
   [junit4]   2> 1129792 INFO  (TEST-TestReplicationHandler.doTestStressReplication-seed#[A17758EE270828B3]) [    ] o.a.s.c.CoreContainer New CoreContainer 425109510
   [junit4]   2> 1129792 INFO  (TEST-TestReplicationHandler.doTestStressReplication-seed#[A17758EE270828B3]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A17758EE270828B3-001/solr-instance-001]
   [junit4]   2> 1129792 WARN  (TEST-TestReplicationHandler.doTestStressReplication-seed#[A17758EE270828B3]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A17758EE270828B3-001/solr-instance-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A17758EE270828B3-001/solr-instance-001/lib
   [junit4]   2> 1129793 INFO  (TEST-TestReplicationHandler.doTestStressReplication-seed#[A17758EE270828B3]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1129794 INFO  (TEST-TestReplicationHandler.doTestStressReplication-seed#[A17758EE270828B3]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1129794 INFO  (TEST-TestReplicationHandler.doTestStressReplication-seed#[A17758EE270828B3]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1129794 INFO  (TEST-TestReplicationHandler.doTestStressReplication-seed#[A17758EE270828B3]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1129794 INFO  (TEST-TestReplicationHandler.doTestStressReplication-seed#[A17758EE270828B3]) [    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1129794 INFO  (TEST-TestReplicationHandler.doTestStressReplication-seed#[A17758EE270828B3]) [    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1129795 INFO  (TEST-TestReplicationHandler.doTestStressReplication-seed#[A17758EE270828B3]) [    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A17758EE270828B3-001/solr-instance-001/.
   [junit4]   2> 1129796 INFO  (TEST-TestReplicationHandler.doTestStressReplication-seed#[A17758EE270828B3]) [    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {loadOnStartup=true, config=solrconfig.xml, transient=false, name=collection1, dataDir=data/, configSetProperties=configsetprops.json, schema=schema.xml}
   [junit4]   2> 1129796 INFO  (TEST-TestReplicationHandler.doTestStressReplication-seed#[A17758EE270828B3]) [    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A17758EE270828B3-001/solr-instance-001/./collection1
   [junit4]   2> 1129797 INFO  (TEST-TestReplicationHandler.doTestStressReplication-seed#[A17758EE270828B3]) [    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 1129797 INFO  (coreLoadExecutor-3225-thread-1) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A17758EE270828B3-001/solr-instance-001/collection1'
   [junit4]   2> 1129797 INFO  (coreLoadExecutor-3225-thread-1) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1129797 INFO  (TEST-TestReplicationHandler.doTestStressReplication-seed#[A17758EE270828B3]) [    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0
   [junit4]   2> 1129797 INFO  (coreLoadExecutor-3225-thread-1) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1129797 INFO  (TEST-TestReplicationHandler.doTestStressReplication-seed#[A17758EE270828B3]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1129805 INFO  (coreLoadExecutor-3225-thread-1) [    ] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 1129818 INFO  (coreLoadExecutor-3225-thread-1) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.6.0
   [junit4]   2> 1129843 INFO  (coreLoadExecutor-3225-thread-1) [    ] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1129848 INFO  (coreLoadExecutor-3225-thread-1) [    ] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1129855 INFO  (coreLoadExecutor-3225-thread-1) [    ] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 1129858 INFO  (coreLoadExecutor-3225-thread-1) [    ] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A17758EE270828B3-001/solr-instance-001/collection1'
   [junit4]   2> 1129858 INFO  (coreLoadExecutor-3225-thread-1) [    ] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A17758EE270828B3-001/solr-instance-001/./collection1
   [junit4]   2> 1129858 INFO  (coreLoadExecutor-3225-thread-1) [    x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 1129858 INFO  (coreLoadExecutor-3225-thread-1) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A17758EE270828B3-001/solr-instance-001/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A17758EE270828B3-001/solr-instance-001/./collection1/data/]
   [junit4]   2> 1129858 INFO  (coreLoadExecutor-3225-thread-1) [    x:collection1] o.a.s.c.SolrCore JMX monitoring not detected for core: collection1
   [junit4]   2> 1129859 INFO  (coreLoadExecutor-3225-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A17758EE270828B3-001/solr-instance-001/./collection1/data
   [junit4]   2> 1129859 INFO  (coreLoadExecutor-3225-thread-1) [    x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A17758EE270828B3-001/solr-instance-001/./collection1/data/index/
   [junit4]   2> 1129859 WARN  (coreLoadExecutor-3225-thread-1) [    x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A17758EE270828B3-001/solr-instance-001/./collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 1129859 INFO  (coreLoadExecutor-3225-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A17758EE270828B3-001/solr-instance-001/./collection1/data/index
   [junit4]   2> 1129859 INFO  (coreLoadExecutor-3225-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=18, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.6263090211337061]
   [junit4]   2> 1129860 INFO  (coreLoadExecutor-3225-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@5ed8a42 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2e0ff618),segFN=segments_1,generation=1}
   [junit4]   2> 1129860 INFO  (coreLoadExecutor-3225-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 1129860 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [    x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A17758EE270828B3-001/solr-instance-001/./collection1/data/
   [junit4]   2> 1129862 INFO  (coreLoadExecutor-3225-thread-1) [    x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 1129864 INFO  (coreLoadExecutor-3225-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1129877 INFO  (coreLoadExecutor-3225-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1129878 INFO  (coreLoadExecutor-3225-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1129879 INFO  (coreLoadExecutor-3225-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1129884 INFO  (coreLoadExecutor-3225-thread-1) [    x:collection1] o.a.s.c.PluginBag requestHandler : 'lazy' created with startup=lazy 
   [junit4]   2> 1129884 INFO  (coreLoadExecutor-3225-thread-1) [    x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/threads,/admin/properties,/update/json,lazy,/admin/logging,/update,/admin/file,/admin/plugins,defaults,standard,/schema,/config,/admin/mbeans,/admin/segments,/update/csv,/update/json/docs,/admin/ping,/replication,/admin/luke,/get,/admin/system
   [junit4]   2> 1129885 INFO  (coreLoadExecutor-3225-thread-1) [    x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 1129886 INFO  (coreLoadExecutor-3225-thread-1) [    x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1129886 INFO  (coreLoadExecutor-3225-thread-1) [    x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1129886 INFO  (coreLoadExecutor-3225-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=33, maxMergeAtOnceExplicit=32, maxMergedSegmentMB=8.455078125, floorSegmentMB=1.513671875, forceMergeDeletesPctAllowed=17.116185295796623, segmentsPerTier=30.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 1129886 INFO  (coreLoadExecutor-3225-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@5ed8a42 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2e0ff618),segFN=segments_1,generation=1}
   [junit4]   2> 1129886 INFO  (coreLoadExecutor-3225-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 1129887 INFO  (coreLoadExecutor-3225-thread-1) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@38626651[collection1] main
   [junit4]   2> 1129887 INFO  (coreLoadExecutor-3225-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A17758EE270828B3-001/solr-instance-001/collection1/conf
   [junit4]   2> 1129887 INFO  (coreLoadExecutor-3225-thread-1) [    x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {storageDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A17758EE270828B3-001/solr-instance-001/collection1/conf}
   [junit4]   2> 1129887 INFO  (coreLoadExecutor-3225-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using file:dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A17758EE270828B3-001/solr-instance-001/collection1/conf
   [junit4]   2> 1129887 INFO  (coreLoadExecutor-3225-thread-1) [    x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 1129887 INFO  (coreLoadExecutor-3225-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler Replication enabled for following config files: schema.xml,xslt/dummy.xsl
   [junit4]   2> 1129887 INFO  (coreLoadExecutor-3225-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1129888 INFO  (searcherExecutor-3226-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@38626651[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1129888 INFO  (coreLoadExecutor-3225-thread-1) [    x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 1130299 INFO  (TEST-TestReplicationHandler.doTestStressReplication-seed#[A17758EE270828B3]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A17758EE270828B3-001/solr-instance-002/collection1
   [junit4]   2> 1130303 INFO  (TEST-TestReplicationHandler.doTestStressReplication-seed#[A17758EE270828B3]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 1130304 INFO  (TEST-TestReplicationHandler.doTestStressReplication-seed#[A17758EE270828B3]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@6d832230{/solr,null,AVAILABLE}
   [junit4]   2> 1130305 INFO  (TEST-TestReplicationHandler.doTestStressReplication-seed#[A17758EE270828B3]) [    ] o.e.j.s.ServerConnector Started ServerConnector@70b9bd20{HTTP/1.1}{127.0.0.1:34055}
   [junit4]   2> 1130305 INFO  (TEST-TestReplicationHandler.doTestStressReplication-seed#[A17758EE270828B3]) [    ] o.e.j.s.Server Started @1133455ms
   [junit4]   2> 1130305 INFO  (TEST-TestReplicationHandler.doTestStressReplication-seed#[A17758EE270828B3]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=34055, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A17758EE270828B3-001/solr-instance-002/collection1/data}
   [junit4]   2> 1130305 INFO  (TEST-TestReplicationHandler.doTestStressReplication-seed#[A17758EE270828B3]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@3827de24
   [junit4]   2> 1130305 INFO  (TEST-TestReplicationHandler.doTestStressReplication-seed#[A17758EE270828B3]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A17758EE270828B3-001/solr-instance-002'
   [junit4]   2> 1130305 INFO  (TEST-TestReplicationHandler.doTestStressReplication-seed#[A17758EE270828B3]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1130305 INFO  (TEST-TestReplicationHandler.doTestStressReplication-seed#[A17758EE270828B3]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1130306 INFO  (TEST-TestReplicationHandler.doTestStressReplication-seed#[A17758EE270828B3]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A17758EE270828B3-001/solr-instance-002/solr.xml
   [junit4]   2> 1130320 INFO  (TEST-TestReplicationHandler.doTestStressReplication-seed#[A17758EE270828B3]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A17758EE270828B3-001/solr-instance-002/.
   [junit4]   2> 1130320 INFO  (TEST-TestReplicationHandler.doTestStressReplication-seed#[A17758EE270828B3]) [    ] o.a.s.c.CoreContainer New CoreContainer 120288616
   [junit4]   2> 1130320 INFO  (TEST-TestReplicationHandler.doTestStressReplication-seed#[A17758EE270828B3]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A17758EE270828B3-001/solr-instance-002]
   [junit4]   2> 1130320 WARN  (TEST-TestReplicationHandler.doTestStressReplication-seed#[A17758EE270828B3]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A17758EE270828B3-001/solr-instance-002/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A17758EE270828B3-001/solr-instance-002/lib
   [junit4]   2> 1130321 INFO  (TEST-TestReplicationHandler.doTestStressReplication-seed#[A17758EE270828B3]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1130322 INFO  (TEST-TestReplicationHandler.doTestStressReplication-seed#[A17758EE270828B3]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1130322 INFO  (TEST-TestReplicationHandler.doTestStressReplication-seed#[A17758EE270828B3]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1130322 INFO  (TEST-TestReplicationHandler.doTestStressReplication-seed#[A17758EE270828B3]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1130323 INFO  (TEST-TestReplicationHandler.doTestStressReplication-seed#[A17758EE270828B3]) [    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1130323 INFO  (TEST-TestReplicationHandler.doTestStressReplication-seed#[A17758EE270828B3]) [    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1130323 INFO  (TEST-TestReplicationHandler.doTestStressReplication-seed#[A17758EE270828B3]) [    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A17758EE270828B3-001/solr-instance-002/.
   [junit4]   2> 1130325 INFO  (TEST-TestReplicationHandler.doTestStressReplication-seed#[A17758EE270828B3]) [    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {transient=false, schema=schema.xml, name=collection1, configSetProperties=configsetprops.json, config=solrconfig.xml, loadOnStartup=true, dataDir=data/}
   [junit4]   2> 1130325 INFO  (TEST-TestReplicationHandler.doTestStressReplication-seed#[A17758EE270828B3]) [    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A17758EE270828B3-001/solr-instance-002/./collection1
   [junit4]   2> 1130325 INFO  (TEST-TestReplicationHandler.doTestStressReplication-seed#[A17758EE270828B3]) [    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 1130325 INFO  (coreLoadExecutor-3235-thread-1) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A17758EE270828B3-001/solr-instance-002/collection1'
   [junit4]   2> 1130325 INFO  (coreLoadExecutor-3235-thread-1) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1130325 INFO  (TEST-TestReplicationHandler.doTestStressReplication-seed#[A17758EE270828B3]) [    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0
   [junit4]   2> 1130325 INFO  (TEST-TestReplicationHandler.doTestStressReplication-seed#[A17758EE270828B3]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1130325 INFO  (coreLoadExecutor-3235-thread-1) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1130333 INFO  (coreLoadExecutor-3235-thread-1) [    ] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 1130345 INFO  (coreLoadExecutor-3235-thread-1) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.6.0
   [junit4]   2> 1130370 INFO  (coreLoadExecutor-3235-thread-1) [    ] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1130374 INFO  (coreLoadExecutor-3235-thread-1) [    ] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1130382 INFO  (coreLoadExecutor-3235-thread-1) [    ] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 1130384 INFO  (coreLoadExecutor-3235-thread-1) [    ] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A17758EE270828B3-001/solr-instance-002/collection1'
   [junit4]   2> 1130384 INFO  (coreLoadExecutor-3235-thread-1) [    ] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A17758EE270828B3-001/solr-instance-002/./collection1
   [junit4]   2> 1130384 INFO  (coreLoadExecutor-3235-thread-1) [    x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 1130385 INFO  (coreLoadExecutor-3235-thread-1) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A17758EE270828B3-001/solr-instance-002/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A17758EE270828B3-001/solr-instance-002/./collection1/data/]
   [junit4]   2> 1130385 INFO  (coreLoadExecutor-3235-thread-1) [    x:collection1] o.a.s.c.SolrCore JMX monitoring not detected for core: collection1
   [junit4]   2> 1130385 INFO  (coreLoadExecutor-3235-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A17758EE270828B3-001/solr-instance-002/./collection1/data
   [junit4]   2> 1130385 INFO  (coreLoadExecutor-3235-thread-1) [    x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A17758EE270828B3-001/solr-instance-002/./collection1/data/index/
   [junit4]   2> 1130385 WARN  (coreLoadExecutor-3235-thread-1) [    x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A17758EE270828B3-001/solr-instance-002/./collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 1130385 INFO  (coreLoadExecutor-3235-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A17758EE270828B3-001/solr-instance-002/./collection1/data/index
   [junit4]   2> 1130386 INFO  (coreLoadExecutor-3235-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=18, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.6263090211337061]
   [junit4]   2> 1130386 INFO  (coreLoadExecutor-3235-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@1cdf9e19 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1d048a1e),segFN=segments_1,generation=1}
   [junit4]   2> 1130386 INFO  (coreLoadExecutor-3235-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 1130387 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [    x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A17758EE270828B3-001/solr-instance-002/./collection1/data/
   [junit4]   2> 1130388 INFO  (coreLoadExecutor-3235-thread-1) [    x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 1130391 INFO  (coreLoadExecutor-3235-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1130392 INFO  (coreLoadExecutor-3235-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1130393 INFO  (coreLoadExecutor-3235-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1130394 INFO  (coreLoadExecutor-3235-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1130396 INFO  (coreLoadExecutor-3235-thread-1) [    x:collection1] o.a.s.c.PluginBag requestHandler : 'lazy' created with startup=lazy 
   [junit4]   2> 1130396 INFO  (coreLoadExecutor-3235-thread-1) [    x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/threads,/admin/properties,/update/json,lazy,/admin/logging,/update,/admin/file,/admin/plugins,defaults,standard,/schema,/config,/admin/mbeans,/admin/segments,/update/csv,/update/json/docs,/admin/ping,/replication,/admin/luke,/get,/admin/system
   [junit4]   2> 1130397 INFO  (coreLoadExecutor-3235-thread-1) [    x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 1130397 INFO  (coreLoadExecutor-3235-thread-1) [    x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1130397 INFO  (coreLoadExecutor-3235-thread-1) [    x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1130397 INFO  (coreLoadExecutor-3235-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=33, maxMergeAtOnceExplicit=32, maxMergedSegmentMB=8.455078125, floorSegmentMB=1.513671875, forceMergeDeletesPctAllowed=17.116185295796623, segmentsPerTier=30.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 1130398 INFO  (coreLoadExecutor-3235-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@1cdf9e19 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1d048a1e),segFN=segments_1,generation=1}
   [junit4]   2> 1130398 INFO  (coreLoadExecutor-3235-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 1130398 INFO  (coreLoadExecutor-3235-thread-1) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@5f746a22[collection1] main
   [junit4]   2> 1130398 INFO  (coreLoadExecutor-3235-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A17758EE270828B3-001/solr-instance-002/collection1/conf
   [junit4]   2> 1130398 INFO  (coreLoadExecutor-3235-thread-1) [    x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {storageDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A17758EE270828B3-001/solr-instance-002/collection1/conf}
   [junit4]   2> 1130398 INFO  (coreLoadExecutor-3235-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using file:dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A17758EE270828B3-001/solr-instance-002/collection1/conf
   [junit4]   2> 1130398 INFO  (coreLoadExecutor-3235-thread-1) [    x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 1130400 INFO  (coreLoadExecutor-3235-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler Poll scheduled at an interval of 1000ms
   [junit4]   2> 1130400 INFO  (searcherExecutor-3236-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@5f746a22[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1130400 INFO  (coreLoadExecutor-3235-thread-1) [    x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 1130828 INFO  (TEST-TestReplicationHandler.doTestStressReplication-seed#[A17758EE270828B3]) [    ] o.e.j.s.ServerConnector Stopped ServerConnector@70b9bd20{HTTP/1.1}{127.0.0.1:0}
   [junit4]   2> 1130828 INFO  (TEST-TestReplicationHandler.doTestStressReplication-seed#[A17758EE270828B3]) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=120288616
   [junit4]   2> 1130829 INFO  (coreCloseExecutor-3241-thread-1) [    x:collection1] o.a.s.c.SolrCore [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@2cacff43
   [junit4]   2> 1130829 INFO  (coreCloseExecutor-3241-thread-1) [    x:collection1] o.a.s.u.DirectUpdateHandler2 closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
   [junit4]   2> 1130829 INFO  (coreCloseExecutor-3241-thread-1) [    x:collection1] o.a.s.u.SolrCoreState Closing SolrCoreState
   [junit4]   2> 1130829 INFO  (coreCloseExecutor-3241-thread-1) [    x:collection1] o.a.s.u.DefaultSolrCoreState SolrCoreState ref count has reached 0 - closing IndexWriter
   [junit4]   2> 1130829 INFO  (coreCloseExecutor-3241-thread-1) [    x:collection1] o.a.s.u.DefaultSolrCoreState closing IndexWriter with IndexWriterCloser
   [junit4]   2> 1130830 INFO  (coreCloseExecutor-3241-thread-1) [    x:collection1] o.a.s.c.SolrCore [collection1] Closing main searcher on request.
   [junit4]   2> 1130830 INFO  (coreCloseExecutor-3241-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing MockDirectoryFactory - 2 directories currently being tracked
   [junit4]   2> 1130830 INFO  (coreCloseExecutor-3241-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory looking to close /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A17758EE270828B3-001/solr-instance-002/./collection1/data/index [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A17758EE270828B3-001/solr-instance-002/./collection1/data/index;done=false>>]
   [junit4]   2> 1130830 INFO  (coreCloseExecutor-3241-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A17758EE270828B3-001/solr-instance-002/./collection1/data/index
   [junit4]   2> 1130830 INFO  (coreCloseExecutor-3241-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory looking to close /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A17758EE270828B3-001/solr-instance-002/./collection1/data [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A17758EE270828B3-001/solr-instance-002/./collection1/data;done=false>>]
   [junit4]   2> 1130830 INFO  (coreCloseExecutor-3241-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A17758EE270828B3-001/solr-instance-002/./collection1/data
   [junit4]   2> 1130831 INFO  (TEST-TestReplicationHandler.doTestStressReplication-seed#[A17758EE270828B3]) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@6d832230{/solr,null,UNAVAILABLE}
   [junit4]   2> 1130832 INFO  (TEST-TestReplicationHandler.doTestStressReplication-seed#[A17758EE270828B3]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 1130836 INFO  (TEST-TestReplicationHandler.doTestStressReplication-seed#[A17758EE270828B3]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@69a28927{/solr,null,AVAILABLE}
   [junit4]   2> 1130836 INFO  (TEST-TestReplicationHandler.doTestStressReplication-seed#[A17758EE270828B3]) [    ] o.e.j.s.ServerConnector Started ServerConnector@6995113d{HTTP/1.1}{127.0.0.1:52930}
   [junit4]   2> 1130836 INFO  (TEST-TestReplicationHandler.doTestStressReplication-seed#[A17758EE270828B3]) [    ] o.e.j.s.Server Started @1133986ms
   [junit4]   2> 1130836 INFO  (TEST-TestReplicationHandler.doTestStressReplication-seed#[A17758EE270828B3]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A17758EE270828B3-001/solr-instance-002/collection1/data, hostPort=52930}
   [junit4]   2> 1130837 INFO  (TEST-TestReplicationHandler.doTestStressReplication-seed#[A17758EE270828B3]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@3827de24
   [junit4]   2> 1130837 INFO  (TEST-TestReplicationHandler.doTestStressReplication-seed#[A17758EE270828B3]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A17758EE270828B3-001/solr-instance-002'
   [junit4]   2> 1130837 INFO  (TEST-TestReplicationHandler.doTestStressReplication-seed#[A17758EE270828B3]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1130837 INFO  (TEST-TestReplicationHandler.doTestStressReplication-seed#[A17758EE270828B3]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1130837 INFO  (TEST-TestReplicationHandler.doTestStressReplication-seed#[A17758EE270828B3]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A17758EE270828B3-001/solr-instance-002/solr.xml
   [junit4]   2> 1130852 INFO  (TEST-TestReplicationHandler.doTestStressReplication-seed#[A17758EE270828B3]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A17758EE270828B3-001/solr-instance-002/.
   [junit4]   2> 1130852 INFO  (TEST-TestReplicationHandler.doTestStressReplication-seed#[A17758EE270828B3]) [    ] o.a.s.c.CoreContainer New CoreContainer 362868542
   [junit4]   2> 1130852 INFO  (TEST-TestReplicationHandler.doTestStressReplication-seed#[A17758EE270828B3]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A17758EE270828B3-001/solr-instance-002]
   [junit4]   2> 1130852 WARN  (TEST-TestReplicationHandler.doTestStressReplication-seed#[A17758EE270828B3]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A17758EE270828B3-001/solr-instance-002/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A17758EE270828B3-001/solr-instance-002/lib
   [junit4]   2> 1130852 INFO  (TEST-TestReplicationHandler.doTestStressReplication-seed#[A17758EE270828B3]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1130854 INFO  (TEST-TestReplicationHandler.doTestStressReplication-seed#[A17758EE270828B3]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1130854 INFO  (TEST-TestReplicationHandler.doTestStressReplication-seed#[A17758EE270828B3]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1130854 INFO  (TEST-TestReplicationHandler.doTestStressReplication-seed#[A17758EE270828B3]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1130854 INFO  (TEST-TestReplicationHandler.doTestStressReplication-seed#[A17758EE270828B3]) [    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1130854 INFO  (TEST-TestReplicationHandler.doTestStressReplication-seed#[A17758EE270828B3]) [    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1130855 INFO  (TEST-TestReplicationHandler.doTestStressReplication-seed#[A17758EE270828B3]) [    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A17758EE270828B3-001/solr-instance-002/.
   [junit4]   2> 1130857 INFO  (TEST-TestReplicationHandler.doTestStressReplication-seed#[A17758EE270828B3]) [    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {dataDir=data/, schema=schema.xml, configSetProperties=configsetprops.json, name=collection1, loadOnStartup=true, config=solrconfig.xml, transient=false}
   [junit4]   2> 1130857 INFO  (TEST-TestReplicationHandler.doTestStressReplication-seed#[A17758EE270828B3]) [    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A17758EE270828B3-001/solr-instance-002/./collection1
   [junit4]   2> 1130857 INFO  (TEST-TestReplicationHandler.doTestStressReplication-seed#[A17758EE270828B3]) [    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 1130857 INFO  (coreLoadExecutor-3247-thread-1) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A17758EE270828B3-001/solr-instance-002/collection1'
   [junit4]   2> 1130857 INFO  (TEST-TestReplicationHandler.doTestStressReplication-seed#[A17758EE270828B3]) [    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0
   [junit4]   2> 1130857 INFO  (coreLoadExecutor-3247-thread-1) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1130857 INFO  (TEST-TestReplicationHandler.doTestStressReplication-seed#[A17758EE270828B3]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1130857 INFO  (coreLoadExecutor-3247-thread-1) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1130866 INFO  (coreLoadExecutor-3247-thread-1) [    ] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 1130879 INFO  (coreLoadExecutor-3247-thread-1) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.6.0
   [junit4]   2> 1130904 INFO  (coreLoadExecutor-3247-thread-1) [    ] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1130908 INFO  (coreLoadExecutor-3247-thread-1) [    ] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1130916 INFO  (coreLoadExecutor-3247-thread-1) [    ] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 1130918 INFO  (coreLoadExecutor-3247-thread-1) [    ] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A17758EE270828B3-001/solr-instance-002/collection1'
   [junit4]   2> 1130918 INFO  (coreLoadExecutor-3247-thread-1) [    ] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A17758EE270828B3-001/solr-instance-002/./collection1
   [junit4]   2> 1130918 INFO  (coreLoadExecutor-3247-thread-1) [    x:collection1] o.a.s.c.SolrCore solr.NRTCachingDirectoryFactory
   [junit4]   2> 1130918 INFO  (coreLoadExecutor-3247-thread-1) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A17758EE270828B3-001/solr-instance-002/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A17758EE270828B3-001/solr-instance-002/./collection1/data/]
   [junit4]   2> 1130918 INFO  (coreLoadExecutor-3247-thread-1) [    x:collection1] o.a.s.c.SolrCore JMX monitoring not detected for core: collection1
   [junit4]   2> 1130919 INFO  (coreLoadExecutor-3247-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A17758EE270828B3-001/solr-instance-002/collection1/data
   [junit4]   2> 1130919 INFO  (coreLoadExecutor-3247-thread-1) [    x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A17758EE270828B3-001/solr-instance-002/./collection1/data/index/
   [junit4]   2> 1130919 WARN  (coreLoadExecutor-3247-thread-1) [    x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A17758EE270828B3-001/solr-instance-002/./collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 1130919 INFO  (coreLoadExecutor-3247-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A17758EE270828B3-001/solr-instance-002/collection1/data/index
   [junit4]   2> 1130920 INFO  (coreLoadExecutor-3247-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=35, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
   [junit4]   2> 1130924 INFO  (coreLoadExecutor-3247-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(MMapDirectory@/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A17758EE270828B3-001/solr-instance-002/collection1/data/index lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@6db0a810; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
   [junit4]   2> 1130924 INFO  (coreLoadExecutor-3247-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 1130925 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [    x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A17758EE270828B3-001/solr-instance-002/./collection1/data/
   [junit4]   2> 1130926 INFO  (coreLoadExecutor-3247-thread-1) [    x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 1130929 INFO  (coreLoadExecutor-3247-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1130930 INFO  (coreLoadExecutor-3247-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1130931 INFO  (coreLoadExecutor-3247-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1130932 INFO  (coreLoadExecutor-3247-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1130934 INFO  (coreLoadExecutor-3247-thread-1) [    x:collection1] o.a.s.c.PluginBag requestHandler : 'lazy' created with startup=lazy 
   [junit4]   2> 1130934 INFO  (coreLoadExecutor-3247-thread-1) [    x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/threads,/admin/properties,/update/json,lazy,/admin/logging,/update,/admin/file,/admin/plugins,defaults,standard,/schema,/config,/admin/mbeans,/admin/segments,/update/csv,/update/json/docs,/admin/ping,/replication,/admin/luke,/get,/admin/system
   [junit4]   2> 1130935 INFO  (coreLoadExecutor-3247-thread-1) [    x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 1130935 INFO  (coreLoadExecutor-3247-thread-1) [    x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1130935 INFO  (coreLoadExecutor-3247-thread-1) [    x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1130936 INFO  (coreLoadExecutor-3247-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=46, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.4016066351153501]
   [junit4]   2> 1130936 INFO  (coreLoadExecutor-3247-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(MMapDirectory@/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A17758EE270828B3-001/solr-instance-002/collection1/data/index lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@6db0a810; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
   [junit4]   2> 1130937 INFO  (coreLoadExecutor-3247-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 1130937 INFO  (coreLoadExecutor-3247-thread-1) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@642689ac[collection1] main
   [junit4]   2> 1130937 INFO  (coreLoadExecutor-3247-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A17758EE270828B3-001/solr-instance-002/collection1/conf
   [junit4]   2> 1130937 INFO  (coreLoadExecutor-3247-thread-1) [    x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {storageDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A17758EE270828B3-001/solr-instance-002/collection1/conf}
   [junit4]   2> 1130937 INFO  (coreLoadExecutor-3247-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using file:dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A17758EE270828B3-001/solr-instance-002/collection1/conf
   [junit4]   2> 1130937 INFO  (coreLoadExecutor-3247-thread-1) [    x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 1130937 INFO  (coreLoadExecutor-3247-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1130938 INFO  (coreLoadExecutor-3247-thread-1) [    x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 1130938 INFO  (searcherExecutor-3248-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@642689ac[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1131361 INFO  (TEST-TestReplicationHandler.doTestStressReplication-seed#[A17758EE270828B3]) [    ] o.e.j.s.ServerConnector Stopped ServerConnector@7ab826d1{HTTP/1.1}{127.0.0.1:0}
   [junit4]   2> 1131361 INFO  (TEST-TestReplicationHandler.doTestStressReplication-seed#[A17758EE270828B3]) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=425109510
   [junit4]   2> 1131361 INFO  (coreCloseExecutor-3252-thread-1) [    x:collection1] o.a.s.c.SolrCore [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@7ef15784
   [junit4]   2> 1131362 INFO  (coreCloseExecutor-3252-thread-1) [    x:collection1] o.a.s.u.DirectUpdateHandler2 closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
   [junit4]   2> 1131362 INFO  (coreCloseExecutor-3252-thread-1) [    x:collection1] o.a.s.u.SolrCoreState Closing SolrCoreState
   [junit4]   2> 1131362 INFO  (coreCloseExecutor-3252-thread-1) [    x:collection1] o.a.s.u.DefaultSolrCoreState SolrCoreState ref count has reached 0 - closing IndexWriter
   [junit4]   2> 1131362 INFO  (coreCloseExecutor-3252-thread-1) [    x:collection1] o.a.s.u.DefaultSolrCoreState closing IndexWriter with IndexWriterCloser
   [junit4]   2> 1131362 INFO  (coreCloseExecutor-3252-thread-1) [    x:collection1] o.a.s.c.SolrCore [collection1] Closing main searcher on request.
   [junit4]   2> 1131362 INFO  (coreCloseExecutor-3252-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing MockDirectoryFactory - 2 directories currently being tracked
   [junit4]   2> 1131362 INFO  (coreCloseExecutor-3252-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory looking to close /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A17758EE270828B3-001/solr-instance-001/./collection1/data/index [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A17758EE270828B3-001/solr-instance-001/./collection1/data/index;done=false>>]
   [junit4]   2> 1131362 INFO  (coreCloseExecutor-3252-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A17758EE270828B3-001/solr-instance-001/./collection1/data/index
   [junit4]   2> 1131362 INFO  (coreCloseExecutor-3252-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory looking to close /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A17758EE270828B3-001/solr-instance-001/./collection1/data [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A17758EE270828B3-001/solr-instance-001/./collection1/data;done=false>>]
   [junit4]   2> 1131363 INFO  (coreCloseExecutor-3252-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A17758EE270828B3-001/solr-instance-001/./collection1/data
   [junit4]   2> 1131363 INFO  (TEST-TestReplicationHandler.doTestStressReplication-seed#[A17758EE270828B3]) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@61be759f{/solr,null,UNAVAILABLE}
   [junit4]   2> 1131365 INFO  (TEST-TestReplicationHandler.doTestStressReplication-seed#[A17758EE270828B3]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 1131366 INFO  (TEST-TestReplicationHandler.doTestStressReplication-seed#[A17758EE270828B3]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@3a56c351{/solr,null,AVAILABLE}
   [junit4]   2> 1131367 INFO  (TEST-TestReplicationHandler.doTestStressReplication-seed#[A17758EE270828B3]) [    ] o.e.j.s.ServerConnector Started ServerConnector@7150b596{HTTP/1.1}{127.0.0.1:44895}
   [junit4]   2> 1131367 INFO  (TEST-TestReplicationHandler.doTestStressReplication-seed#[A17758EE270828B3]) [    ] o.e.j.s.Server Started @1134517ms
   [junit4]   2> 1131367 INFO  (TEST-TestReplicationHandler.doTestStressReplication-seed#[A17758EE270828B3]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A17758EE270828B3-001/solr-instance-001/collection1/data, hostPort=44895, hostContext=/solr}
   [junit4]   2> 1131367 INFO  (TEST-TestReplicationHandler.doTestStressReplication-seed#[A17758EE270828B3]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@3827de24
   [junit4]   2> 1131367 INFO  (TEST-TestReplicationHandler.doTestStressReplication-seed#[A17758EE270828B3]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A17758EE270828B3-001/solr-instance-001'
   [junit4]   2> 1131367 INFO  (TEST-TestReplicationHandler.doTestStressReplication-seed#[A17758EE270828B3]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1131367 INFO  (TEST-TestReplicationHandler.doTestStressReplication-seed#[A17758EE270828B3]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1131368 INFO  (TEST-TestReplicationHandler.doTestStressReplication-seed#[A17758EE270828B3]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A17758EE270828B3-001/solr-instance-001/solr.xml
   [junit4]   2> 1131382 INFO  (TEST-TestReplicationHandler.doTestStressReplication-seed#[A17758EE270828B3]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A17758EE270828B3-001/solr-instance-001/.
   [junit4]   2> 1131382 INFO  (TEST-TestReplicationHandler.doTestStressReplication-seed#[A17758EE270828B3]) [    ] o.a.s.c.CoreContainer New CoreContainer 378475896
   [junit4]   2> 1131382 INFO  (TEST-TestReplicationHandler.doTestStressReplication-seed#[A17758EE270828B3]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A17758EE270828B3-001/solr-instance-001]
   [junit4]   2> 1131382 WARN  (TEST-TestReplicationHandler.doTestStressReplication-seed#[A17758EE270828B3]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A17758EE270828B3-001/solr-instance-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A17758EE270828B3-001/solr-instance-001/lib
   [junit4]   2> 1131382 INFO  (TEST-TestReplicationHandler.doTestStressReplication-seed#[A17758EE270828B3]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1131383 INFO  (TEST-TestReplicationHandler.doTestStressReplication-seed#[A17758EE270828B3]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1131384 INFO  (TEST-TestReplicationHandler.doTestStressReplication-seed#[A17758EE270828B3]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1131384 INFO  (TEST-TestReplicationHandler.doTestStressReplication-seed#[A17758EE270828B3]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1131384 INFO  (TEST-TestReplicationHandler.doTestStressReplication-seed#[A17758EE270828B3]) [    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1131384 INFO  (TEST-TestReplicationHandler.doTestStressReplication-seed#[A17758EE270828B3]) [    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1131385 INFO  (TEST-TestReplicationHandler.doTestStressReplication-seed#[A17758EE270828B3]) [    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A17758EE270828B3-001/solr-instance-001/.
   [junit4]   2> 1131386 INFO  (TEST-TestReplicationHandler.doTestStressReplication-seed#[A17758EE270828B3]) [    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {config=solrconfig.xml, dataDir=data/, name=collection1, loadOnStartup=true, schema=schema.xml, transient=false, configSetProperties=configsetprops.json}
   [junit4]   2> 1131386 INFO  (TEST-TestReplicationHandler.doTestStressReplication-seed#[A17758EE270828B3]) [    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A17758EE270828B3-001/solr-instance-001/./collection1
   [junit4]   2> 1131386 INFO  (TEST-TestReplicationHandler.doTestStressReplication-seed#[A17758EE270828B3]) [    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 1131387 INFO  (coreLoadExecutor-3258-thread-1) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A17758EE270828B3-001/solr-instance-001/collection1'
   [junit4]   2> 1131387 INFO  (coreLoadExecutor-3258-thread-1) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1131387 INFO  (coreLoadExecutor-3258-thread-1) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1131394 INFO  (coreLoadExecutor-3258-thread-1) [    ] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 1131401 INFO  (TEST-TestReplicationHandler.doTestStressReplication-seed#[A17758EE270828B3]) [    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0
   [junit4]   2> 1131401 INFO  (TEST-TestReplicationHandler.doTestStressReplication-seed#[A17758EE270828B3]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1131407 INFO  (coreLoadExecutor-3258-thread-1) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.6.0
   [junit4]   2> 1131448 INFO  (coreLoadExecutor-3258-thread-1) [    ] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1131452 INFO  (coreLoadExecutor-3258-thread-1) [    ] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1131460 INFO  (coreLoadExecutor-3258-thread-1) [    ] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 1131462 INFO  (coreLoadExecutor-3258-thread-1) [    ] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A17758EE270828B3-001/solr-instance-001/collection1'
   [junit4]   2> 1131462 INFO  (coreLoadExecutor-3258-thread-1) [    ] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A17758EE270828B3-001/solr-instance-001/./collection1
   [junit4]   2> 1131462 INFO  (coreLoadExecutor-3258-thread-1) [    x:collection1] o.a.s.c.SolrCore solr.NRTCachingDirectoryFactory
   [junit4]   2> 1131462 INFO  (coreLoadExecutor-3258-thread-1) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A17758EE270828B3-001/solr-instance-001/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A17758EE270828B3-001/solr-instance-001/./collection1/data/]
   [junit4]   2> 1131462 INFO  (coreLoadExecutor-3258-thread-1) [    x:collection1] o.a.s.c.SolrCore JMX monitoring not detected for core: collection1
   [junit4]   2> 1131462 INFO  (coreLoadExecutor-3258-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A17758EE270828B3-001/solr-instance-001/collection1/data
   [junit4]   2> 1131463 INFO  (coreLoadExecutor-3258-thread-1) [    x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A17758EE270828B3-001/solr-instance-001/./collection1/data/index/
   [junit4]   2> 1131463 WARN  (coreLoadExecutor-3258-thread-1) [    x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A17758EE270828B3-001/solr-instance-001/./collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 1131463 INFO  (coreLoadExecutor-3258-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A17758EE270828B3-001/solr-instance-001/collection1/data/index
   [junit4]   2> 1131463 INFO  (coreLoadExecutor-3258-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=35, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
   [junit4]   2> 1131469 INFO  (coreLoadExecutor-3258-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(MMapDirectory@/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A17758EE270828B3-001/solr-instance-001/collection1/data/index lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5459c07d; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
   [junit4]   2> 1131469 INFO  (coreLoadExecutor-3258-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 1131470 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [    x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A17758EE270828B3-001/solr-instance-001/./collection1/data/
   [junit4]   2> 1131471 INFO  (coreLoadExecutor-3258-thread-1) [    x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 1131474 INFO  (coreLoadExecutor-3258-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1131475 INFO  (coreLoadExecutor-3258-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1131476 INFO  (coreLoadExecutor-3258-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1131477 INFO  (coreLoadExecutor-3258-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1131479 INFO  (coreLoadExecutor-3258-thread-1) [    x:collection1] o.a.s.c.PluginBag requestHandler : 'lazy' created with startup=lazy 
   [junit4]   2> 1131479 INFO  (coreLoadExecutor-3258-thread-1) [    x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/threads,/admin/properties,/update/json,lazy,/admin/logging,/update,/admin/file,/admin/plugins,defaults,standard,/schema,/config,/admin/mbeans,/admin/segments,/update/csv,/update/json/docs,/admin/ping,/replication,/admin/luke,/get,/admin/system
   [junit4]   2> 1131480 INFO  (coreLoadExecutor-3258-thread-1) [    x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 1131480 INFO  (coreLoadExecutor-3258-thread-1) [    x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1131480 INFO  (coreLoadExecutor-3258-thread-1) [    x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1131481 INFO  (coreLoadExecutor-3258-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=46, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.4016066351153501]
   [junit4]   2> 1131481 INFO  (coreLoadExecutor-3258-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(MMapDirectory@/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A17758EE270828B3-001/solr-instance-001/collection1/data/index lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5459c07d; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
   [junit4]   2> 1131481 INFO  (coreLoadExecutor-3258-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 1131482 INFO  (coreLoadExecutor-3258-thread-1) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@4987e31f[collection1] main
   [junit4]   2> 1131482 INFO  (coreLoadExecutor-3258-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A17758EE270828B3-001/solr-instance-001/collection1/conf
   [junit4]   2> 1131482 INFO  (coreLoadExecutor-3258-thread-1) [    x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {storageDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A17758EE270828B3-001/solr-instance-001/collection1/conf}
   [junit4]   2> 1131482 INFO  (coreLoadExecutor-3258-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using file:dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A17758EE270828B3-001/solr-instance-001/collection1/conf
   [junit4]   2> 1131482 INFO  (coreLoadExecutor-3258-thread-1) [    x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 1131482 INFO  (coreLoadExecutor-3258-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler Replication enabled for following config files: schema.xml
   [junit4]   2> 1131482 INFO  (coreLoadExecutor-3258-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1131483 INFO  (searcherExecutor-3259-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@4987e31f[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1131483 INFO  (coreLoadExecutor-3258-thread-1) [    x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 1131904 INFO  (qtp1810677227-16630) [    x:collection1] o.a.s.u.DirectUpdateHandler2 [collection1] REMOVING ALL DOCUMENTS FROM INDEX
   [junit4]   2> 1131905 INFO  (qtp1810677227-16630) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={version=2&wt=javabin}{deleteByQuery=*:*} 0 0
   [junit4]   2> 1131906 INFO  (qtp299939495-16608) [    x:collection1] o.a.s.u.DirectUpdateHandler2 [collection1] REMOVING ALL DOCUMENTS FROM INDEX
   [junit4]   2> 1131907 INFO  (qtp299939495-16608) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={version=2&wt=javabin}{deleteByQuery=*:*} 0 0
   [junit4]   2> 1131908 INFO  (qtp299939495-16611) [    x:collection1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 1131914 INFO  (qtp299939495-16611) [    x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 	commit{dir=NRTCachingDirectory(MMapDirectory@/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A17758EE270828B3-001/solr-instance-002/collection1/data/index lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@6db0a810; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
   [junit4]   2> 	commit{dir=NRTCachingDirectory(MMapDirectory@/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A17758EE270828B3-001/solr-instance-002/collection1/data/index lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@6db0a810; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2}
   [junit4]   2> 1131914 INFO  (qtp299939495-16611) [    x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 2
   [junit4]   2> 1131916 INFO  (qtp299939495-16611) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@2142eae6[collection1] main
   [junit4]   2> 1131916 INFO  (qtp299939495-16611) [    x:collection1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 1131916 INFO  (searcherExecutor-3248-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@2142eae6[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1131916 INFO  (qtp299939495-16611) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={waitSearcher=true&commit=true&version=2&softCommit=false&wt=javabin}{commit=} 0 7
   [junit4]   2> 1131920 INFO  (qtp1810677227-16631) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={version=2&wt=javabin}{add=[0]} 0 1
   [junit4]   2> 1131922 INFO  (qtp1810677227-16632) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={version=2&wt=javabin}{add=[1]} 0 0
   [junit4]   2> 1131924 INFO  (qtp1810677227-16633) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={version=2&wt=javabin}{add=[2]} 0 0
   [junit4]   2> 1131926 INFO  (qtp1810677227-16634) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={version=2&wt=javabin}{add=[3]} 0 0
 

[...truncated too long message...]

enkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-ldif-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-mavibot-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-protocol-kerberos-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-protocol-ldap-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-protocol-shared-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-xdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/api-all-1.0.0-M20.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/bcprov-jdk15-1.45.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/cglib-nodep-2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/commons-collections-3.2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/commons-math3-3.4.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/easymock-3.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/ehcache-core-2.4.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/hadoop-common-2.6.0-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/hadoop-hdfs-2.6.0-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/hadoop-minikdc-2.6.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jackson-annotations-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jackson-databind-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jersey-core-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jersey-server-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jetty-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jetty-util-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/mina-core-2.0.0-M5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/objenesis-1.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/analysis/icu/lucene-analyzers-icu-5.6.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-analysis-extras/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/queryparser/classes/test:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/contrib/analysis-extras/lib/icu4j-54.1.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-launcher.jar:/x1/jenkins/.ant/lib/ivy-2.3.0.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-antlr.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-regexp.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-resolver.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-net.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jmf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-oro.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-xalan2.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-logging.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jsch.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit4.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-javamail.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jai.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jdepend.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bsf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-swing.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-netrexx.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bcel.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-testutil.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-log4j.jar:/x1/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.7/lib/tools.jar:/x1/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.3.2.jar com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J1-20160213_092621_820.events @/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J1-20160213_092621_820.suites -stdin
   [junit4] ERROR: JVM J1 ended with an exception: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1501)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:121)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:955)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:952)
   [junit4] 	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4] 	at java.lang.Thread.run(Thread.java:745)

BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:757: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:694: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:534: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1477: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1033: At least one slave process threw an exception, first: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.

Total time: 205 minutes 32 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
ERROR: Failed to archive {README.txt=README.txt, solr/build/solr-core/test/temp/junit4-J0-20160213_092621_820.events=solr/build/solr-core/test/temp/junit4-J0-20160213_092621_820.events, solr/build/solr-core/test/temp/junit4-J2-20160213_092621_824.events=solr/build/solr-core/test/temp/junit4-J2-20160213_092621_824.events, heapdumps/java_pid16312.hprof=heapdumps/java_pid16312.hprof, solr/build/solr-core/test/temp/junit4-J1-20160213_092621_820.events=solr/build/solr-core/test/temp/junit4-J1-20160213_092621_820.events} due to internal error; falling back to full archiving
java.lang.NullPointerException
	at jsync.protocol.ChangeInputStream.read(ChangeInputStream.java:69)
	at com.cloudbees.jenkins.plugins.jsync.archiver.MD5DigestingInputStream.read(MD5DigestingInputStream.java:39)
	at com.google.common.io.LimitInputStream.read(LimitInputStream.java:79)
	at java.io.FilterInputStream.read(FilterInputStream.java:107)
	at com.google.common.io.ByteStreams.copy(ByteStreams.java:193)
	at jsync.protocol.FileSequenceReader.read(FileSequenceReader.java:35)
	at com.cloudbees.jenkins.plugins.jsync.archiver.JSyncArtifactManager.remoteSync(JSyncArtifactManager.java:148)
	at com.cloudbees.jenkins.plugins.jsync.archiver.JSyncArtifactManager.archive(JSyncArtifactManager.java:72)
	at hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:219)
	at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:74)
	at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
	at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:776)
	at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:723)
	at hudson.model.Build$BuildExecution.post2(Build.java:183)
	at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:670)
	at hudson.model.Run.execute(Run.java:1763)
	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
	at hudson.model.ResourceController.execute(ResourceController.java:98)
	at hudson.model.Executor.run(Executor.java:381)
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



Re: [JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1095 - Still Failing

Posted by Nicholas Knize <nk...@gmail.com>.
I've got a fix.

On Thursday, February 11, 2016, Apache Jenkins Server <
jenkins@builds.apache.org> wrote:

> Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1095/
>
> 1 tests failed.
> FAILED:
> org.apache.lucene.spatial.geopoint.search.TestGeoPointQuery.testRandomBig
>
> Error Message:
> Java heap space
>
> Stack Trace:
> java.lang.OutOfMemoryError: Java heap space
>         at
> __randomizedtesting.SeedInfo.seed([48C5189314C6B77E:CF92651C859FCBFE]:0)
>         at org.apache.lucene.util.ArrayUtil.grow(ArrayUtil.java:354)
>         at
> org.apache.lucene.codecs.memory.DirectPostingsFormat$DirectField.<init>(DirectPostingsFormat.java:363)
>         at
> org.apache.lucene.codecs.memory.DirectPostingsFormat$DirectFields.<init>(DirectPostingsFormat.java:131)
>         at
> org.apache.lucene.codecs.memory.DirectPostingsFormat.fieldsProducer(DirectPostingsFormat.java:115)
>         at
> org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsReader.<init>(PerFieldPostingsFormat.java:261)
>         at
> org.apache.lucene.codecs.perfield.PerFieldPostingsFormat.fieldsProducer(PerFieldPostingsFormat.java:341)
>         at
> org.apache.lucene.index.SegmentCoreReaders.<init>(SegmentCoreReaders.java:104)
>         at
> org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:65)
>         at
> org.apache.lucene.index.CheckIndex.checkIndex(CheckIndex.java:649)
>         at org.apache.lucene.util.TestUtil.checkIndex(TestUtil.java:277)
>         at
> org.apache.lucene.store.MockDirectoryWrapper.close(MockDirectoryWrapper.java:757)
>         at org.apache.lucene.util.IOUtils.close(IOUtils.java:97)
>         at org.apache.lucene.util.IOUtils.close(IOUtils.java:84)
>         at
> org.apache.lucene.spatial.util.BaseGeoPointTestCase.verify(BaseGeoPointTestCase.java:757)
>         at
> org.apache.lucene.spatial.util.BaseGeoPointTestCase.doTestRandom(BaseGeoPointTestCase.java:405)
>         at
> org.apache.lucene.spatial.util.BaseGeoPointTestCase.testRandomBig(BaseGeoPointTestCase.java:333)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>         at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>         at java.lang.reflect.Method.invoke(Method.java:606)
>         at
> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
>         at
> com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
>         at
> com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
>         at
> com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
>         at
> org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
>         at
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
>         at
> org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
>         at
> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
>         at
> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
>         at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>         at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
>         at
> com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
>
>
>
>
> Build Log:
> [...truncated 9380 lines...]
>    [junit4] Suite:
> org.apache.lucene.spatial.geopoint.search.TestGeoPointQuery
>    [junit4]   2> NOTE: download the large Jenkins line-docs file by
> running 'ant get-jenkins-line-docs' in the lucene directory.
>    [junit4]   2> NOTE: reproduce with: ant test
> -Dtestcase=TestGeoPointQuery -Dtests.method=testRandomBig
> -Dtests.seed=48C5189314C6B77E -Dtests.multiplier=2 -Dtests.nightly=true
> -Dtests.slow=true
> -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt
> -Dtests.locale=es-DO -Dtests.timezone=US/Samoa -Dtests.asserts=true
> -Dtests.file.encoding=UTF-8
>    [junit4] ERROR   1197s J2 | TestGeoPointQuery.testRandomBig <<<
>    [junit4]    > Throwable #1: java.lang.OutOfMemoryError: Java heap space
>    [junit4]    >        at
> __randomizedtesting.SeedInfo.seed([48C5189314C6B77E:CF92651C859FCBFE]:0)
>    [junit4]    >        at
> org.apache.lucene.util.ArrayUtil.grow(ArrayUtil.java:354)
>    [junit4]    >        at
> org.apache.lucene.codecs.memory.DirectPostingsFormat$DirectField.<init>(DirectPostingsFormat.java:363)
>    [junit4]    >        at
> org.apache.lucene.codecs.memory.DirectPostingsFormat$DirectFields.<init>(DirectPostingsFormat.java:131)
>    [junit4]    >        at
> org.apache.lucene.codecs.memory.DirectPostingsFormat.fieldsProducer(DirectPostingsFormat.java:115)
>    [junit4]    >        at
> org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsReader.<init>(PerFieldPostingsFormat.java:261)
>    [junit4]    >        at
> org.apache.lucene.codecs.perfield.PerFieldPostingsFormat.fieldsProducer(PerFieldPostingsFormat.java:341)
>    [junit4]    >        at
> org.apache.lucene.index.SegmentCoreReaders.<init>(SegmentCoreReaders.java:104)
>    [junit4]    >        at
> org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:65)
>    [junit4]    >        at
> org.apache.lucene.index.CheckIndex.checkIndex(CheckIndex.java:649)
>    [junit4]    >        at
> org.apache.lucene.util.TestUtil.checkIndex(TestUtil.java:277)
>    [junit4]    >        at
> org.apache.lucene.store.MockDirectoryWrapper.close(MockDirectoryWrapper.java:757)
>    [junit4]    >        at
> org.apache.lucene.util.IOUtils.close(IOUtils.java:97)
>    [junit4]    >        at
> org.apache.lucene.util.IOUtils.close(IOUtils.java:84)
>    [junit4]    >        at
> org.apache.lucene.spatial.util.BaseGeoPointTestCase.verify(BaseGeoPointTestCase.java:757)
>    [junit4]    >        at
> org.apache.lucene.spatial.util.BaseGeoPointTestCase.doTestRandom(BaseGeoPointTestCase.java:405)
>    [junit4]    >        at
> org.apache.lucene.spatial.util.BaseGeoPointTestCase.testRandomBig(BaseGeoPointTestCase.java:333)
>    [junit4]   2> NOTE: leaving temporary files on disk at:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/spatial/test/J2/temp/lucene.spatial.geopoint.search.TestGeoPointQuery_48C5189314C6B77E-001
>    [junit4]   2> NOTE: test params are: codec=Asserting(Lucene54):
> {point=PostingsFormat(name=Direct),
> id=PostingsFormat(name=LuceneVarGapDocFreqInterval),
> string=PostingsFormat(name=LuceneFixedGap)},
> docValues:{point=DocValuesFormat(name=Asserting),
> id=DocValuesFormat(name=Memory)},
> sim=RandomSimilarity(queryNorm=false,coord=no): {}, locale=es-DO,
> timezone=US/Samoa
>    [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation
> 1.7.0_80 (64-bit)/cpus=4,threads=1,free=274326384,total=376963072
>    [junit4]   2> NOTE: All tests run in this JVM: [PortedSolr3Test,
> DateNRStrategyTest, Geo3dRptTest, TestGeoPointQuery]
>    [junit4] Completed [25/25 (1!)] on J2 in 1276.30s, 25 tests, 1 error
> <<< FAILURES!
>
> [...truncated 1 lines...]
>    [junit4] JVM J2: stdout was not empty, see:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/spatial/test/temp/junit4-J2-20160212_004803_794.sysout
>    [junit4] >>> JVM J2 emitted unexpected output (verbatim) ----
>    [junit4] java.lang.OutOfMemoryError: Java heap space
>    [junit4] Dumping heap to
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/heapdumps/java_pid30185.hprof
> ...
>    [junit4] Heap dump file created [557517084 bytes in 4.633 secs]
>    [junit4] <<< JVM J2: EOF ----
>
> [...truncated 10 lines...]
> BUILD FAILED
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:757:
> The following error occurred while executing this line:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:694:
> The following error occurred while executing this line:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59:
> The following error occurred while executing this line:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build.xml:475:
> The following error occurred while executing this line:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:2273:
> The following error occurred while executing this line:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/module-build.xml:58:
> The following error occurred while executing this line:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1477:
> The following error occurred while executing this line:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1033:
> There were test failures: 25 suites, 269 tests, 1 error, 11 ignored (10
> assumptions) [seed: 48C5189314C6B77E]
>
> Total time: 150 minutes 57 seconds
> Build step 'Invoke Ant' marked build as failure
> Archiving artifacts
> Recording test results
> Email was triggered for: Failure - Any
> Sending email for trigger: Failure - Any
>
>
>

[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1095 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1095/

1 tests failed.
FAILED:  org.apache.lucene.spatial.geopoint.search.TestGeoPointQuery.testRandomBig

Error Message:
Java heap space

Stack Trace:
java.lang.OutOfMemoryError: Java heap space
	at __randomizedtesting.SeedInfo.seed([48C5189314C6B77E:CF92651C859FCBFE]:0)
	at org.apache.lucene.util.ArrayUtil.grow(ArrayUtil.java:354)
	at org.apache.lucene.codecs.memory.DirectPostingsFormat$DirectField.<init>(DirectPostingsFormat.java:363)
	at org.apache.lucene.codecs.memory.DirectPostingsFormat$DirectFields.<init>(DirectPostingsFormat.java:131)
	at org.apache.lucene.codecs.memory.DirectPostingsFormat.fieldsProducer(DirectPostingsFormat.java:115)
	at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsReader.<init>(PerFieldPostingsFormat.java:261)
	at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat.fieldsProducer(PerFieldPostingsFormat.java:341)
	at org.apache.lucene.index.SegmentCoreReaders.<init>(SegmentCoreReaders.java:104)
	at org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:65)
	at org.apache.lucene.index.CheckIndex.checkIndex(CheckIndex.java:649)
	at org.apache.lucene.util.TestUtil.checkIndex(TestUtil.java:277)
	at org.apache.lucene.store.MockDirectoryWrapper.close(MockDirectoryWrapper.java:757)
	at org.apache.lucene.util.IOUtils.close(IOUtils.java:97)
	at org.apache.lucene.util.IOUtils.close(IOUtils.java:84)
	at org.apache.lucene.spatial.util.BaseGeoPointTestCase.verify(BaseGeoPointTestCase.java:757)
	at org.apache.lucene.spatial.util.BaseGeoPointTestCase.doTestRandom(BaseGeoPointTestCase.java:405)
	at org.apache.lucene.spatial.util.BaseGeoPointTestCase.testRandomBig(BaseGeoPointTestCase.java:333)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)




Build Log:
[...truncated 9380 lines...]
   [junit4] Suite: org.apache.lucene.spatial.geopoint.search.TestGeoPointQuery
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestGeoPointQuery -Dtests.method=testRandomBig -Dtests.seed=48C5189314C6B77E -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=es-DO -Dtests.timezone=US/Samoa -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR   1197s J2 | TestGeoPointQuery.testRandomBig <<<
   [junit4]    > Throwable #1: java.lang.OutOfMemoryError: Java heap space
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([48C5189314C6B77E:CF92651C859FCBFE]:0)
   [junit4]    > 	at org.apache.lucene.util.ArrayUtil.grow(ArrayUtil.java:354)
   [junit4]    > 	at org.apache.lucene.codecs.memory.DirectPostingsFormat$DirectField.<init>(DirectPostingsFormat.java:363)
   [junit4]    > 	at org.apache.lucene.codecs.memory.DirectPostingsFormat$DirectFields.<init>(DirectPostingsFormat.java:131)
   [junit4]    > 	at org.apache.lucene.codecs.memory.DirectPostingsFormat.fieldsProducer(DirectPostingsFormat.java:115)
   [junit4]    > 	at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsReader.<init>(PerFieldPostingsFormat.java:261)
   [junit4]    > 	at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat.fieldsProducer(PerFieldPostingsFormat.java:341)
   [junit4]    > 	at org.apache.lucene.index.SegmentCoreReaders.<init>(SegmentCoreReaders.java:104)
   [junit4]    > 	at org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:65)
   [junit4]    > 	at org.apache.lucene.index.CheckIndex.checkIndex(CheckIndex.java:649)
   [junit4]    > 	at org.apache.lucene.util.TestUtil.checkIndex(TestUtil.java:277)
   [junit4]    > 	at org.apache.lucene.store.MockDirectoryWrapper.close(MockDirectoryWrapper.java:757)
   [junit4]    > 	at org.apache.lucene.util.IOUtils.close(IOUtils.java:97)
   [junit4]    > 	at org.apache.lucene.util.IOUtils.close(IOUtils.java:84)
   [junit4]    > 	at org.apache.lucene.spatial.util.BaseGeoPointTestCase.verify(BaseGeoPointTestCase.java:757)
   [junit4]    > 	at org.apache.lucene.spatial.util.BaseGeoPointTestCase.doTestRandom(BaseGeoPointTestCase.java:405)
   [junit4]    > 	at org.apache.lucene.spatial.util.BaseGeoPointTestCase.testRandomBig(BaseGeoPointTestCase.java:333)
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/spatial/test/J2/temp/lucene.spatial.geopoint.search.TestGeoPointQuery_48C5189314C6B77E-001
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene54): {point=PostingsFormat(name=Direct), id=PostingsFormat(name=LuceneVarGapDocFreqInterval), string=PostingsFormat(name=LuceneFixedGap)}, docValues:{point=DocValuesFormat(name=Asserting), id=DocValuesFormat(name=Memory)}, sim=RandomSimilarity(queryNorm=false,coord=no): {}, locale=es-DO, timezone=US/Samoa
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_80 (64-bit)/cpus=4,threads=1,free=274326384,total=376963072
   [junit4]   2> NOTE: All tests run in this JVM: [PortedSolr3Test, DateNRStrategyTest, Geo3dRptTest, TestGeoPointQuery]
   [junit4] Completed [25/25 (1!)] on J2 in 1276.30s, 25 tests, 1 error <<< FAILURES!

[...truncated 1 lines...]
   [junit4] JVM J2: stdout was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/spatial/test/temp/junit4-J2-20160212_004803_794.sysout
   [junit4] >>> JVM J2 emitted unexpected output (verbatim) ----
   [junit4] java.lang.OutOfMemoryError: Java heap space
   [junit4] Dumping heap to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/heapdumps/java_pid30185.hprof ...
   [junit4] Heap dump file created [557517084 bytes in 4.633 secs]
   [junit4] <<< JVM J2: EOF ----

[...truncated 10 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:757: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:694: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build.xml:475: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:2273: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/module-build.xml:58: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1477: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1033: There were test failures: 25 suites, 269 tests, 1 error, 11 ignored (10 assumptions) [seed: 48C5189314C6B77E]

Total time: 150 minutes 57 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1094 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1094/

11 tests failed.
FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Test abandoned because suite timeout was reached.

Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
	at __randomizedtesting.SeedInfo.seed([6FFCD939E547DA5A]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.CollectionsAPIDistributedZkTest

Error Message:
Java heap space

Stack Trace:
java.lang.OutOfMemoryError: Java heap space


FAILED:  org.apache.solr.cloud.DeleteShardTest.test

Error Message:
java.util.concurrent.TimeoutException: Could not connect to ZooKeeper 127.0.0.1:44687 within 45000 ms

Stack Trace:
org.apache.solr.common.SolrException: java.util.concurrent.TimeoutException: Could not connect to ZooKeeper 127.0.0.1:44687 within 45000 ms
	at __randomizedtesting.SeedInfo.seed([6FFCD939E547DA5A:E7A8E6E34BBBB7A2]:0)
	at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:181)
	at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:115)
	at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:105)
	at org.apache.solr.cloud.AbstractZkTestCase.buildZooKeeper(AbstractZkTestCase.java:88)
	at org.apache.solr.cloud.AbstractZkTestCase.buildZooKeeper(AbstractZkTestCase.java:82)
	at org.apache.solr.cloud.AbstractDistribZkTestBase.distribSetUp(AbstractDistribZkTestBase.java:75)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.distribSetUp(AbstractFullDistribZkTestBase.java:197)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:934)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.util.concurrent.TimeoutException: Could not connect to ZooKeeper 127.0.0.1:44687 within 45000 ms
	at org.apache.solr.common.cloud.ConnectionManager.waitForConnected(ConnectionManager.java:208)
	at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:173)
	... 38 more


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.DeleteShardTest

Error Message:
9 threads leaked from SUITE scope at org.apache.solr.cloud.DeleteShardTest:     1) Thread[id=11133, name=qtp1297997705-11133, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)         at java.lang.Thread.run(Thread.java:745)    2) Thread[id=11134, name=qtp370483280-11134, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)         at java.lang.Thread.run(Thread.java:745)    3) Thread[id=11131, name=qtp415401506-11131, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)         at java.lang.Thread.run(Thread.java:745)    4) Thread[id=11070, name=NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0, state=RUNNABLE, group=TGRP-DeleteShardTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)         at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:178)         at java.lang.Thread.run(Thread.java:745)    5) Thread[id=11132, name=qtp1868655190-11132, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)         at java.lang.Thread.run(Thread.java:745)    6) Thread[id=11073, name=ProcessThread(sid:0 cport:-1):, state=WAITING, group=TGRP-DeleteShardTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at org.apache.zookeeper.server.PrepRequestProcessor.run(PrepRequestProcessor.java:120)    7) Thread[id=11069, name=Thread-9987, state=WAITING, group=TGRP-DeleteShardTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:834)         at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:867)         at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1197)         at java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.lock(ReentrantReadWriteLock.java:945)         at com.sun.jmx.mbeanserver.Repository.addMBean(Repository.java:416)         at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerWithRepository(DefaultMBeanServerInterceptor.java:1898)         at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:966)         at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:900)         at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:324)         at com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:522)         at org.apache.zookeeper.jmx.MBeanRegistry.register(MBeanRegistry.java:96)         at org.apache.zookeeper.server.ZooKeeperServer.registerJMX(ZooKeeperServer.java:377)         at org.apache.zookeeper.server.ZooKeeperServer.startup(ZooKeeperServer.java:410)         at org.apache.zookeeper.server.NIOServerCnxnFactory.startup(NIOServerCnxnFactory.java:123)         at org.apache.solr.cloud.ZkTestServer$ZKServerMain.runFromConfig(ZkTestServer.java:309)         at org.apache.solr.cloud.ZkTestServer$2.run(ZkTestServer.java:491)    8) Thread[id=11135, name=qtp413760287-11135, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)         at java.lang.Thread.run(Thread.java:745)    9) Thread[id=11071, name=SessionTracker, state=TIMED_WAITING, group=TGRP-DeleteShardTest]         at java.lang.Object.wait(Native Method)         at org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:146)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 9 threads leaked from SUITE scope at org.apache.solr.cloud.DeleteShardTest: 
   1) Thread[id=11133, name=qtp1297997705-11133, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)
        at java.lang.Thread.run(Thread.java:745)
   2) Thread[id=11134, name=qtp370483280-11134, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)
        at java.lang.Thread.run(Thread.java:745)
   3) Thread[id=11131, name=qtp415401506-11131, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)
        at java.lang.Thread.run(Thread.java:745)
   4) Thread[id=11070, name=NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0, state=RUNNABLE, group=TGRP-DeleteShardTest]
        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
        at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
        at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:178)
        at java.lang.Thread.run(Thread.java:745)
   5) Thread[id=11132, name=qtp1868655190-11132, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)
        at java.lang.Thread.run(Thread.java:745)
   6) Thread[id=11073, name=ProcessThread(sid:0 cport:-1):, state=WAITING, group=TGRP-DeleteShardTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at org.apache.zookeeper.server.PrepRequestProcessor.run(PrepRequestProcessor.java:120)
   7) Thread[id=11069, name=Thread-9987, state=WAITING, group=TGRP-DeleteShardTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:834)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:867)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1197)
        at java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.lock(ReentrantReadWriteLock.java:945)
        at com.sun.jmx.mbeanserver.Repository.addMBean(Repository.java:416)
        at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerWithRepository(DefaultMBeanServerInterceptor.java:1898)
        at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:966)
        at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:900)
        at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:324)
        at com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:522)
        at org.apache.zookeeper.jmx.MBeanRegistry.register(MBeanRegistry.java:96)
        at org.apache.zookeeper.server.ZooKeeperServer.registerJMX(ZooKeeperServer.java:377)
        at org.apache.zookeeper.server.ZooKeeperServer.startup(ZooKeeperServer.java:410)
        at org.apache.zookeeper.server.NIOServerCnxnFactory.startup(NIOServerCnxnFactory.java:123)
        at org.apache.solr.cloud.ZkTestServer$ZKServerMain.runFromConfig(ZkTestServer.java:309)
        at org.apache.solr.cloud.ZkTestServer$2.run(ZkTestServer.java:491)
   8) Thread[id=11135, name=qtp413760287-11135, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)
        at java.lang.Thread.run(Thread.java:745)
   9) Thread[id=11071, name=SessionTracker, state=TIMED_WAITING, group=TGRP-DeleteShardTest]
        at java.lang.Object.wait(Native Method)
        at org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:146)
	at __randomizedtesting.SeedInfo.seed([6FFCD939E547DA5A]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.DeleteShardTest

Error Message:
There are still zombie threads that couldn't be terminated:    1) Thread[id=11070, name=NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0, state=RUNNABLE, group=TGRP-DeleteShardTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)         at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:178)         at java.lang.Thread.run(Thread.java:745)    2) Thread[id=11069, name=Thread-9987, state=WAITING, group=TGRP-DeleteShardTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:834)         at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:867)         at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1197)         at java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.lock(ReentrantReadWriteLock.java:945)         at com.sun.jmx.mbeanserver.Repository.addMBean(Repository.java:416)         at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerWithRepository(DefaultMBeanServerInterceptor.java:1898)         at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:966)         at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:900)         at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:324)         at com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:522)         at org.apache.zookeeper.jmx.MBeanRegistry.register(MBeanRegistry.java:96)         at org.apache.zookeeper.server.ZooKeeperServer.registerJMX(ZooKeeperServer.java:377)         at org.apache.zookeeper.server.ZooKeeperServer.startup(ZooKeeperServer.java:410)         at org.apache.zookeeper.server.NIOServerCnxnFactory.startup(NIOServerCnxnFactory.java:123)         at org.apache.solr.cloud.ZkTestServer$ZKServerMain.runFromConfig(ZkTestServer.java:309)         at org.apache.solr.cloud.ZkTestServer$2.run(ZkTestServer.java:491)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   1) Thread[id=11070, name=NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0, state=RUNNABLE, group=TGRP-DeleteShardTest]
        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
        at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
        at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:178)
        at java.lang.Thread.run(Thread.java:745)
   2) Thread[id=11069, name=Thread-9987, state=WAITING, group=TGRP-DeleteShardTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:834)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:867)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1197)
        at java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.lock(ReentrantReadWriteLock.java:945)
        at com.sun.jmx.mbeanserver.Repository.addMBean(Repository.java:416)
        at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerWithRepository(DefaultMBeanServerInterceptor.java:1898)
        at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:966)
        at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:900)
        at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:324)
        at com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:522)
        at org.apache.zookeeper.jmx.MBeanRegistry.register(MBeanRegistry.java:96)
        at org.apache.zookeeper.server.ZooKeeperServer.registerJMX(ZooKeeperServer.java:377)
        at org.apache.zookeeper.server.ZooKeeperServer.startup(ZooKeeperServer.java:410)
        at org.apache.zookeeper.server.NIOServerCnxnFactory.startup(NIOServerCnxnFactory.java:123)
        at org.apache.solr.cloud.ZkTestServer$ZKServerMain.runFromConfig(ZkTestServer.java:309)
        at org.apache.solr.cloud.ZkTestServer$2.run(ZkTestServer.java:491)
	at __randomizedtesting.SeedInfo.seed([6FFCD939E547DA5A]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.core.QueryResultKeyTest

Error Message:


Stack Trace:
java.lang.AssertionError
	at org.apache.lucene.util.TestRuleTemporaryFilesCleanup.before(TestRuleTemporaryFilesCleanup.java:112)
	at com.carrotsearch.randomizedtesting.rules.TestRuleAdapter$1.before(TestRuleAdapter.java:26)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:35)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  junit.framework.TestSuite.org.apache.solr.core.QueryResultKeyTest

Error Message:
5 threads leaked from SUITE scope at org.apache.solr.core.QueryResultKeyTest:     1) Thread[id=10712, name=qtp1868655190-10712, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)         at java.lang.Thread.run(Thread.java:745)    2) Thread[id=10710, name=qtp370483280-10710, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)         at java.lang.Thread.run(Thread.java:745)    3) Thread[id=10713, name=qtp1297997705-10713, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)         at java.lang.Thread.run(Thread.java:745)    4) Thread[id=10711, name=qtp415401506-10711, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)         at java.lang.Thread.run(Thread.java:745)    5) Thread[id=10714, name=qtp413760287-10714, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)         at java.lang.Thread.run(Thread.java:745)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 5 threads leaked from SUITE scope at org.apache.solr.core.QueryResultKeyTest: 
   1) Thread[id=10712, name=qtp1868655190-10712, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)
        at java.lang.Thread.run(Thread.java:745)
   2) Thread[id=10710, name=qtp370483280-10710, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)
        at java.lang.Thread.run(Thread.java:745)
   3) Thread[id=10713, name=qtp1297997705-10713, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)
        at java.lang.Thread.run(Thread.java:745)
   4) Thread[id=10711, name=qtp415401506-10711, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)
        at java.lang.Thread.run(Thread.java:745)
   5) Thread[id=10714, name=qtp413760287-10714, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)
        at java.lang.Thread.run(Thread.java:745)
	at __randomizedtesting.SeedInfo.seed([6FFCD939E547DA5A]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.core.TestNRTOpen

Error Message:
5 threads leaked from SUITE scope at org.apache.solr.core.TestNRTOpen:     1) Thread[id=11056, name=qtp415401506-11056, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)         at java.lang.Thread.run(Thread.java:745)    2) Thread[id=11055, name=qtp370483280-11055, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)         at java.lang.Thread.run(Thread.java:745)    3) Thread[id=11054, name=qtp1297997705-11054, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)         at java.lang.Thread.run(Thread.java:745)    4) Thread[id=11052, name=qtp1868655190-11052, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)         at java.lang.Thread.run(Thread.java:745)    5) Thread[id=11053, name=qtp413760287-11053, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)         at java.lang.Thread.run(Thread.java:745)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 5 threads leaked from SUITE scope at org.apache.solr.core.TestNRTOpen: 
   1) Thread[id=11056, name=qtp415401506-11056, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)
        at java.lang.Thread.run(Thread.java:745)
   2) Thread[id=11055, name=qtp370483280-11055, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)
        at java.lang.Thread.run(Thread.java:745)
   3) Thread[id=11054, name=qtp1297997705-11054, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)
        at java.lang.Thread.run(Thread.java:745)
   4) Thread[id=11052, name=qtp1868655190-11052, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)
        at java.lang.Thread.run(Thread.java:745)
   5) Thread[id=11053, name=qtp413760287-11053, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)
        at java.lang.Thread.run(Thread.java:745)
	at __randomizedtesting.SeedInfo.seed([6FFCD939E547DA5A]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.core.TestQuerySenderListener

Error Message:


Stack Trace:
java.lang.AssertionError
	at __randomizedtesting.SeedInfo.seed([6FFCD939E547DA5A]:0)
	at org.apache.lucene.util.TestRuleRestoreSystemProperties.before(TestRuleRestoreSystemProperties.java:52)
	at com.carrotsearch.randomizedtesting.rules.TestRuleAdapter$1.before(TestRuleAdapter.java:26)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:35)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  junit.framework.TestSuite.org.apache.solr.spelling.WordBreakSolrSpellCheckerTest

Error Message:
ObjectTracker found 643 object(s) that were not released!!! [MockDirectoryWrapper, SolrCore, SolrCore, SolrCore, SolrCore, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, SolrCore, SolrCore, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, SolrCore, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, SolrCore, MockDirectoryWrapper, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, SolrCore, MDCAwareThreadPoolExecutor, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, SolrCore, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, SolrCore, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, SolrCore, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MockDirectoryWrapper, SolrCore, MockDirectoryWrapper, SolrCore, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MDCAwareThreadPoolExecutor, MDCAwareThreadPoolExecutor, SolrCore, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MockDirectoryWrapper, SolrCore, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, SolrCore, SolrCore, SolrCore, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, SolrCore, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, SolrCore, MockDirectoryWrapper, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, SolrCore, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MDCAwareThreadPoolExecutor, SolrCore, MockDirectoryWrapper, SolrCore, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MockDirectoryWrapper, SolrCore, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, SolrCore, MockDirectoryWrapper, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, SolrCore, MockDirectoryWrapper, SolrCore, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MockDirectoryWrapper, SolrCore, MockDirectoryWrapper, SolrCore, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, SolrCore, SolrCore, MockDirectoryWrapper, SolrCore, SolrCore, MockDirectoryWrapper, SolrCore, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, SolrCore, MockDirectoryWrapper, SolrCore, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, SolrCore, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, SolrCore, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MockDirectoryWrapper, SolrCore, SolrCore, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, SolrCore, MockDirectoryWrapper, SolrCore, MDCAwareThreadPoolExecutor, SolrCore, MockDirectoryWrapper, SolrCore, SolrCore, MockDirectoryWrapper, SolrCore, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, SolrCore, MockDirectoryWrapper, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, SolrCore, MDCAwareThreadPoolExecutor, SolrCore, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MDCAwareThreadPoolExecutor, MDCAwareThreadPoolExecutor, MDCAwareThreadPoolExecutor, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, SolrCore, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, SolrCore, SolrZkClient, MockDirectoryWrapper, SolrCore, SolrCore, MockDirectoryWrapper, SolrCore, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, SolrCore, SolrCore, MockDirectoryWrapper, MockDirectoryWrapper, SolrCore, MDCAwareThreadPoolExecutor, MDCAwareThreadPoolExecutor, SolrCore, MDCAwareThreadPoolExecutor, SolrCore, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, SolrCore, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MDCAwareThreadPoolExecutor, MDCAwareThreadPoolExecutor, SolrCore, SolrCore, MockDirectoryWrapper, MockDirectoryWrapper, SolrCore, SolrCore, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, SolrCore, SolrCore, MDCAwareThreadPoolExecutor, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MDCAwareThreadPoolExecutor, SolrCore, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, SolrCore, SolrCore, SolrCore, MockDirectoryWrapper, SolrCore, MockDirectoryWrapper, SolrCore, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, SolrCore, SolrCore, SolrCore, SolrCore, MockDirectoryWrapper, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, SolrCore, SolrZkClient, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, SolrCore, MockDirectoryWrapper, MockDirectoryWrapper, SolrCore, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, SolrCore, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MDCAwareThreadPoolExecutor, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MockDirectoryWrapper, SolrCore, MockDirectoryWrapper, MockDirectoryWrapper, SolrCore, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, SolrCore, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, SolrCore, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MDCAwareThreadPoolExecutor, SolrCore, MDCAwareThreadPoolExecutor, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MockDirectoryWrapper, SolrCore, SolrCore, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, SolrCore, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, SolrCore, MDCAwareThreadPoolExecutor, SolrCore, MockDirectoryWrapper, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, SolrCore, MDCAwareThreadPoolExecutor, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, SolrCore, SolrCore, MockDirectoryWrapper, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, SolrCore, SolrCore, MDCAwareThreadPoolExecutor, MDCAwareThreadPoolExecutor, SolrCore, MDCAwareThreadPoolExecutor, SolrCore, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, SolrCore, MockDirectoryWrapper, SolrCore, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, SolrCore, MDCAwareThreadPoolExecutor, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MockDirectoryWrapper, SolrCore, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, SolrCore, SolrCore, SolrCore, MockDirectoryWrapper, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MDCAwareThreadPoolExecutor, SolrCore, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, SolrCore, MDCAwareThreadPoolExecutor, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, SolrCore, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, SolrCore, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, SolrCore, MockDirectoryWrapper, SolrCore, MDCAwareThreadPoolExecutor, SolrCore, SolrCore, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, RawDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, SolrZkClient, MDCAwareThreadPoolExecutor, MDCAwareThreadPoolExecutor, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MockDirectoryWrapper, SolrCore, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, SolrCore, MDCAwareThreadPoolExecutor, SolrCore, MDCAwareThreadPoolExecutor, SolrCore, SolrCore, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, SolrCore, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, SolrCore, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, SolrCore, SolrCore, SolrCore, MDCAwareThreadPoolExecutor, MDCAwareThreadPoolExecutor, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, SolrZkClient, MockDirectoryWrapper, SolrCore, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MockDirectoryWrapper, SolrCore, SolrCore, SolrCore, SolrCore, MDCAwareThreadPoolExecutor, MDCAwareThreadPoolExecutor, SolrZkClient, MockDirectoryWrapper, SolrCore, SolrCore, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MDCAwareThreadPoolExecutor, SolrCore, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, SolrCore, MockDirectoryWrapper, MockDirectoryWrapper, SolrCore, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, SolrCore]

Stack Trace:
java.lang.AssertionError: ObjectTracker found 643 object(s) that were not released!!! [MockDirectoryWrapper, SolrCore, SolrCore, SolrCore, SolrCore, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, SolrCore, SolrCore, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, SolrCore, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, SolrCore, MockDirectoryWrapper, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, SolrCore, MDCAwareThreadPoolExecutor, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, SolrCore, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, SolrCore, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, SolrCore, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MockDirectoryWrapper, SolrCore, MockDirectoryWrapper, SolrCore, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MDCAwareThreadPoolExecutor, MDCAwareThreadPoolExecutor, SolrCore, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MockDirectoryWrapper, SolrCore, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, SolrCore, SolrCore, SolrCore, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, SolrCore, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, SolrCore, MockDirectoryWrapper, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, SolrCore, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MDCAwareThreadPoolExecutor, SolrCore, MockDirectoryWrapper, SolrCore, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MockDirectoryWrapper, SolrCore, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, SolrCore, MockDirectoryWrapper, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, SolrCore, MockDirectoryWrapper, SolrCore, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MockDirectoryWrapper, SolrCore, MockDirectoryWrapper, SolrCore, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, SolrCore, SolrCore, MockDirectoryWrapper, SolrCore, SolrCore, MockDirectoryWrapper, SolrCore, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, SolrCore, MockDirectoryWrapper, SolrCore, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, SolrCore, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, SolrCore, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MockDirectoryWrapper, SolrCore, SolrCore, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, SolrCore, MockDirectoryWrapper, SolrCore, MDCAwareThreadPoolExecutor, SolrCore, MockDirectoryWrapper, SolrCore, SolrCore, MockDirectoryWrapper, SolrCore, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, SolrCore, MockDirectoryWrapper, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, SolrCore, MDCAwareThreadPoolExecutor, SolrCore, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MDCAwareThreadPoolExecutor, MDCAwareThreadPoolExecutor, MDCAwareThreadPoolExecutor, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, SolrCore, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, SolrCore, SolrZkClient, MockDirectoryWrapper, SolrCore, SolrCore, MockDirectoryWrapper, SolrCore, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, SolrCore, SolrCore, MockDirectoryWrapper, MockDirectoryWrapper, SolrCore, MDCAwareThreadPoolExecutor, MDCAwareThreadPoolExecutor, SolrCore, MDCAwareThreadPoolExecutor, SolrCore, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, SolrCore, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MDCAwareThreadPoolExecutor, MDCAwareThreadPoolExecutor, SolrCore, SolrCore, MockDirectoryWrapper, MockDirectoryWrapper, SolrCore, SolrCore, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, SolrCore, SolrCore, MDCAwareThreadPoolExecutor, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MDCAwareThreadPoolExecutor, SolrCore, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, SolrCore, SolrCore, SolrCore, MockDirectoryWrapper, SolrCore, MockDirectoryWrapper, SolrCore, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, SolrCore, SolrCore, SolrCore, SolrCore, MockDirectoryWrapper, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, SolrCore, SolrZkClient, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, SolrCore, MockDirectoryWrapper, MockDirectoryWrapper, SolrCore, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, SolrCore, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MDCAwareThreadPoolExecutor, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MockDirectoryWrapper, SolrCore, MockDirectoryWrapper, MockDirectoryWrapper, SolrCore, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, SolrCore, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, SolrCore, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MDCAwareThreadPoolExecutor, SolrCore, MDCAwareThreadPoolExecutor, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MockDirectoryWrapper, SolrCore, SolrCore, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, SolrCore, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, SolrCore, MDCAwareThreadPoolExecutor, SolrCore, MockDirectoryWrapper, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, SolrCore, MDCAwareThreadPoolExecutor, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, SolrCore, SolrCore, MockDirectoryWrapper, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, SolrCore, SolrCore, MDCAwareThreadPoolExecutor, MDCAwareThreadPoolExecutor, SolrCore, MDCAwareThreadPoolExecutor, SolrCore, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, SolrCore, MockDirectoryWrapper, SolrCore, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, SolrCore, MDCAwareThreadPoolExecutor, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MockDirectoryWrapper, SolrCore, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, SolrCore, SolrCore, SolrCore, MockDirectoryWrapper, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MDCAwareThreadPoolExecutor, SolrCore, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, SolrCore, MDCAwareThreadPoolExecutor, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, SolrCore, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, SolrCore, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, SolrCore, MockDirectoryWrapper, SolrCore, MDCAwareThreadPoolExecutor, SolrCore, SolrCore, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, RawDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, SolrZkClient, MDCAwareThreadPoolExecutor, MDCAwareThreadPoolExecutor, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MockDirectoryWrapper, SolrCore, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, SolrCore, MDCAwareThreadPoolExecutor, SolrCore, MDCAwareThreadPoolExecutor, SolrCore, SolrCore, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, SolrCore, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, SolrCore, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, SolrCore, SolrCore, SolrCore, MDCAwareThreadPoolExecutor, MDCAwareThreadPoolExecutor, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, SolrZkClient, MockDirectoryWrapper, SolrCore, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MockDirectoryWrapper, SolrCore, SolrCore, SolrCore, SolrCore, MDCAwareThreadPoolExecutor, MDCAwareThreadPoolExecutor, SolrZkClient, MockDirectoryWrapper, SolrCore, SolrCore, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MDCAwareThreadPoolExecutor, SolrCore, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, SolrCore, MockDirectoryWrapper, MockDirectoryWrapper, SolrCore, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, SolrCore]
	at __randomizedtesting.SeedInfo.seed([6FFCD939E547DA5A]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.junit.Assert.assertNull(Assert.java:551)
	at org.apache.solr.SolrTestCaseJ4.afterClass(SolrTestCaseJ4.java:228)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:834)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  junit.framework.TestSuite.org.apache.solr.spelling.WordBreakSolrSpellCheckerTest

Error Message:
5 threads leaked from SUITE scope at org.apache.solr.spelling.WordBreakSolrSpellCheckerTest:     1) Thread[id=10895, name=qtp415401506-10895, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)         at java.lang.Thread.run(Thread.java:745)    2) Thread[id=10897, name=qtp1868655190-10897, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)         at java.lang.Thread.run(Thread.java:745)    3) Thread[id=10898, name=qtp1297997705-10898, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)         at java.lang.Thread.run(Thread.java:745)    4) Thread[id=10899, name=qtp370483280-10899, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)         at java.lang.Thread.run(Thread.java:745)    5) Thread[id=10896, name=qtp413760287-10896, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)         at java.lang.Thread.run(Thread.java:745)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 5 threads leaked from SUITE scope at org.apache.solr.spelling.WordBreakSolrSpellCheckerTest: 
   1) Thread[id=10895, name=qtp415401506-10895, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)
        at java.lang.Thread.run(Thread.java:745)
   2) Thread[id=10897, name=qtp1868655190-10897, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)
        at java.lang.Thread.run(Thread.java:745)
   3) Thread[id=10898, name=qtp1297997705-10898, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)
        at java.lang.Thread.run(Thread.java:745)
   4) Thread[id=10899, name=qtp370483280-10899, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)
        at java.lang.Thread.run(Thread.java:745)
   5) Thread[id=10896, name=qtp413760287-10896, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)
        at java.lang.Thread.run(Thread.java:745)
	at __randomizedtesting.SeedInfo.seed([6FFCD939E547DA5A]:0)




Build Log:
[...truncated 12631 lines...]
   [junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_6FFCD939E547DA5A-001/init-core-data-001
   [junit4]   2> 135720 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[6FFCD939E547DA5A]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false)
   [junit4]   2> 135720 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[6FFCD939E547DA5A]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /ii_qf/f
   [junit4]   2> 135728 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6FFCD939E547DA5A]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 135728 INFO  (Thread-2356) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 135728 INFO  (Thread-2356) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 135828 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6FFCD939E547DA5A]) [    ] o.a.s.c.ZkTestServer start zk server on port:43346
   [junit4]   2> 135829 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6FFCD939E547DA5A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 135829 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6FFCD939E547DA5A]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 135839 INFO  (zkCallback-56-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6e1cf1b5 name:ZooKeeperConnection Watcher:127.0.0.1:43346 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 135839 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6FFCD939E547DA5A]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 135840 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6FFCD939E547DA5A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 135840 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6FFCD939E547DA5A]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 135845 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x152ceddf0410000, likely client has closed socket
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 135846 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6FFCD939E547DA5A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 135849 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6FFCD939E547DA5A]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 135855 INFO  (zkCallback-57-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@11902c05 name:ZooKeeperConnection Watcher:127.0.0.1:43346/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 135855 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6FFCD939E547DA5A]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 135856 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6FFCD939E547DA5A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 135856 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6FFCD939E547DA5A]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 135859 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6FFCD939E547DA5A]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 135861 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6FFCD939E547DA5A]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 135863 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6FFCD939E547DA5A]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 135865 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6FFCD939E547DA5A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 135865 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6FFCD939E547DA5A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 135868 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6FFCD939E547DA5A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 135869 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6FFCD939E547DA5A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 135871 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6FFCD939E547DA5A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 135872 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6FFCD939E547DA5A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 135874 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6FFCD939E547DA5A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 135874 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6FFCD939E547DA5A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 135876 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6FFCD939E547DA5A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 135877 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6FFCD939E547DA5A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 135879 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6FFCD939E547DA5A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 135879 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6FFCD939E547DA5A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 135882 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6FFCD939E547DA5A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/j

[...truncated too long message...]

bPoll(QueuedThreadPool.java:531)
   [junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)
   [junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >    6) Thread[id=11073, name=ProcessThread(sid:0 cport:-1):, state=WAITING, group=TGRP-DeleteShardTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
   [junit4]    >         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]    >         at org.apache.zookeeper.server.PrepRequestProcessor.run(PrepRequestProcessor.java:120)
   [junit4]    >    7) Thread[id=11069, name=Thread-9987, state=WAITING, group=TGRP-DeleteShardTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:834)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:867)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1197)
   [junit4]    >         at java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.lock(ReentrantReadWriteLock.java:945)
   [junit4]    >         at com.sun.jmx.mbeanserver.Repository.addMBean(Repository.java:416)
   [junit4]    >         at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerWithRepository(DefaultMBeanServerInterceptor.java:1898)
   [junit4]    >         at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:966)
   [junit4]    >         at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:900)
   [junit4]    >         at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:324)
   [junit4]    >         at com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:522)
   [junit4]    >         at org.apache.zookeeper.jmx.MBeanRegistry.register(MBeanRegistry.java:96)
   [junit4]    >         at org.apache.zookeeper.server.ZooKeeperServer.registerJMX(ZooKeeperServer.java:377)
   [junit4]    >         at org.apache.zookeeper.server.ZooKeeperServer.startup(ZooKeeperServer.java:410)
   [junit4]    >         at org.apache.zookeeper.server.NIOServerCnxnFactory.startup(NIOServerCnxnFactory.java:123)
   [junit4]    >         at org.apache.solr.cloud.ZkTestServer$ZKServerMain.runFromConfig(ZkTestServer.java:309)
   [junit4]    >         at org.apache.solr.cloud.ZkTestServer$2.run(ZkTestServer.java:491)
   [junit4]    >    8) Thread[id=11135, name=qtp413760287-11135, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
   [junit4]    >         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)
   [junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)
   [junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)
   [junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >    9) Thread[id=11071, name=SessionTracker, state=TIMED_WAITING, group=TGRP-DeleteShardTest]
   [junit4]    >         at java.lang.Object.wait(Native Method)
   [junit4]    >         at org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:146)
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([6FFCD939E547DA5A]:0)Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   [junit4]    >    1) Thread[id=11070, name=NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0, state=RUNNABLE, group=TGRP-DeleteShardTest]
   [junit4]    >         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
   [junit4]    >         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
   [junit4]    >         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)
   [junit4]    >         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
   [junit4]    >         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
   [junit4]    >         at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:178)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >    2) Thread[id=11069, name=Thread-9987, state=WAITING, group=TGRP-DeleteShardTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:834)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:867)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1197)
   [junit4]    >         at java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.lock(ReentrantReadWriteLock.java:945)
   [junit4]    >         at com.sun.jmx.mbeanserver.Repository.addMBean(Repository.java:416)
   [junit4]    >         at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerWithRepository(DefaultMBeanServerInterceptor.java:1898)
   [junit4]    >         at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:966)
   [junit4]    >         at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:900)
   [junit4]    >         at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:324)
   [junit4]    >         at com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:522)
   [junit4]    >         at org.apache.zookeeper.jmx.MBeanRegistry.register(MBeanRegistry.java:96)
   [junit4]    >         at org.apache.zookeeper.server.ZooKeeperServer.registerJMX(ZooKeeperServer.java:377)
   [junit4]    >         at org.apache.zookeeper.server.ZooKeeperServer.startup(ZooKeeperServer.java:410)
   [junit4]    >         at org.apache.zookeeper.server.NIOServerCnxnFactory.startup(NIOServerCnxnFactory.java:123)
   [junit4]    >         at org.apache.solr.cloud.ZkTestServer$ZKServerMain.runFromConfig(ZkTestServer.java:309)
   [junit4]    >         at org.apache.solr.cloud.ZkTestServer$2.run(ZkTestServer.java:491)
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([6FFCD939E547DA5A]:0)
   [junit4] Completed [486/568 (6!)] on J1 in 73.66s, 1 test, 3 errors <<< FAILURES!

[...truncated 247 lines...]
   [junit4] JVM J1: stdout was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J1-20160211_054418_957.sysout
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] java.lang.OutOfMemoryError: GC overhead limit exceeded
   [junit4] Dumping heap to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/heapdumps/java_pid7918.hprof ...
   [junit4] Heap dump file created [624519490 bytes in 6.639 secs]
   [junit4] <<< JVM J1: EOF ----

   [junit4] JVM J1: stderr was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J1-20160211_054418_957.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] 
   [junit4] Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "NIOServerCxn.Factory:0.0.0.0/0.0.0.0:43346"
   [junit4] <<< JVM J1: EOF ----

[...truncated 17 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:757: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:694: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:534: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1477: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1033: There were test failures: 568 suites (7 ignored), 2269 tests, 9 suite-level errors, 2 errors, 283 ignored (10 assumptions) [seed: 6FFCD939E547DA5A]

Total time: 252 minutes 8 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1093 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1093/

1 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest

Error Message:
ObjectTracker found 1 object(s) that were not released!!! [HdfsTransactionLog]

Stack Trace:
java.lang.AssertionError: ObjectTracker found 1 object(s) that were not released!!! [HdfsTransactionLog]
	at __randomizedtesting.SeedInfo.seed([D3F26B27978BDE60]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.junit.Assert.assertNull(Assert.java:551)
	at org.apache.solr.SolrTestCaseJ4.afterClass(SolrTestCaseJ4.java:228)
	at sun.reflect.GeneratedMethodAccessor21.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:834)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 11301 lines...]
   [junit4] Suite: org.apache.solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest_D3F26B27978BDE60-001/init-core-data-001
   [junit4]   2> 472261 INFO  (SUITE-HdfsChaosMonkeyNothingIsSafeTest-seed#[D3F26B27978BDE60]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   2> 472924 WARN  (SUITE-HdfsChaosMonkeyNothingIsSafeTest-seed#[D3F26B27978BDE60]-worker) [    ] o.a.h.u.NativeCodeLoader Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
   [junit4]   1> Formatting using clusterid: testClusterID
   [junit4]   2> 473853 WARN  (SUITE-HdfsChaosMonkeyNothingIsSafeTest-seed#[D3F26B27978BDE60]-worker) [    ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
   [junit4]   2> 474071 INFO  (SUITE-HdfsChaosMonkeyNothingIsSafeTest-seed#[D3F26B27978BDE60]-worker) [    ] o.m.log Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
   [junit4]   2> 474083 WARN  (SUITE-HdfsChaosMonkeyNothingIsSafeTest-seed#[D3F26B27978BDE60]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 474185 INFO  (SUITE-HdfsChaosMonkeyNothingIsSafeTest-seed#[D3F26B27978BDE60]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 474250 INFO  (SUITE-HdfsChaosMonkeyNothingIsSafeTest-seed#[D3F26B27978BDE60]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/hdfs to ./temp/Jetty_localhost_48911_hdfs____.pvpzr5/webapp
   [junit4]   2> 474482 INFO  (SUITE-HdfsChaosMonkeyNothingIsSafeTest-seed#[D3F26B27978BDE60]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 475233 INFO  (SUITE-HdfsChaosMonkeyNothingIsSafeTest-seed#[D3F26B27978BDE60]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:48911
   [junit4]   2> 476101 WARN  (SUITE-HdfsChaosMonkeyNothingIsSafeTest-seed#[D3F26B27978BDE60]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 476112 INFO  (SUITE-HdfsChaosMonkeyNothingIsSafeTest-seed#[D3F26B27978BDE60]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 476135 INFO  (SUITE-HdfsChaosMonkeyNothingIsSafeTest-seed#[D3F26B27978BDE60]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_52971_datanode____.xg5amb/webapp
   [junit4]   2> 476287 INFO  (SUITE-HdfsChaosMonkeyNothingIsSafeTest-seed#[D3F26B27978BDE60]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 481619 INFO  (SUITE-HdfsChaosMonkeyNothingIsSafeTest-seed#[D3F26B27978BDE60]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:52971
   [junit4]   2> 481732 WARN  (SUITE-HdfsChaosMonkeyNothingIsSafeTest-seed#[D3F26B27978BDE60]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 481735 INFO  (SUITE-HdfsChaosMonkeyNothingIsSafeTest-seed#[D3F26B27978BDE60]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 481752 INFO  (SUITE-HdfsChaosMonkeyNothingIsSafeTest-seed#[D3F26B27978BDE60]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_57034_datanode____2yfaja/webapp
   [junit4]   2> 481883 INFO  (SUITE-HdfsChaosMonkeyNothingIsSafeTest-seed#[D3F26B27978BDE60]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 482181 INFO  (SUITE-HdfsChaosMonkeyNothingIsSafeTest-seed#[D3F26B27978BDE60]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:57034
   [junit4]   2> 482668 INFO  (IPC Server handler 8 on 50452) [    ] BlockStateChange BLOCK* processReport: from storage DS-4da58444-b95f-4e3f-9a63-2cf8ccd2332f node DatanodeRegistration(127.0.0.1, datanodeUuid=652eb657-033e-402d-aff6-24a590660598, infoPort=57034, ipcPort=36094, storageInfo=lv=-56;cid=testClusterID;nsid=1380035281;c=0), blocks: 0, hasStaleStorages: true, processing time: 3 msecs
   [junit4]   2> 482668 INFO  (IPC Server handler 8 on 50452) [    ] BlockStateChange BLOCK* processReport: from storage DS-4d012d88-f9a9-4c1c-8fa3-31efe120e0e0 node DatanodeRegistration(127.0.0.1, datanodeUuid=652eb657-033e-402d-aff6-24a590660598, infoPort=57034, ipcPort=36094, storageInfo=lv=-56;cid=testClusterID;nsid=1380035281;c=0), blocks: 0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 482668 INFO  (IPC Server handler 4 on 50452) [    ] BlockStateChange BLOCK* processReport: from storage DS-6c33d942-3867-45c1-b1aa-26001407115b node DatanodeRegistration(127.0.0.1, datanodeUuid=4653d1e9-bfb5-4ce6-abac-302fc5aafe0d, infoPort=52971, ipcPort=47647, storageInfo=lv=-56;cid=testClusterID;nsid=1380035281;c=0), blocks: 0, hasStaleStorages: true, processing time: 0 msecs
   [junit4]   2> 482669 INFO  (IPC Server handler 4 on 50452) [    ] BlockStateChange BLOCK* processReport: from storage DS-cccbbdd3-078b-4a67-bb0f-867068117c9b node DatanodeRegistration(127.0.0.1, datanodeUuid=4653d1e9-bfb5-4ce6-abac-302fc5aafe0d, infoPort=52971, ipcPort=47647, storageInfo=lv=-56;cid=testClusterID;nsid=1380035281;c=0), blocks: 0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 483072 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 483073 INFO  (Thread-1705) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 483073 INFO  (Thread-1705) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 483173 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [    ] o.a.s.c.ZkTestServer start zk server on port:41962
   [junit4]   2> 483173 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 483174 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 483176 INFO  (zkCallback-509-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@50247cb4 name:ZooKeeperConnection Watcher:127.0.0.1:41962 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 483176 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 483177 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 483177 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 483180 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x152c86327470000, likely client has closed socket
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 483180 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 483181 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 483182 INFO  (zkCallback-510-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@34bd7e7e name:ZooKeeperConnection Watcher:127.0.0.1:41962/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 483182 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 483182 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 483182 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 483184 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 483186 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 483187 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 483189 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 483189 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 483192 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
   [junit4]   2> 483192 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 483194 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 483194 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 483196 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 483196 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 483198 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 483198 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 483200 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 483200 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 483202 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 483202 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 483204 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 483204 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 483206 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 483206 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 483208 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 483208 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 483210 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 483210 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 483329 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest_D3F26B27978BDE60-001/control-001/cores/collection1
   [junit4]   2> 483331 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 483333 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@676a61d4{/,null,AVAILABLE}
   [junit4]   2> 483333 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [    ] o.e.j.s.ServerConnector Started ServerConnector@5f978bd1{HTTP/1.1}{127.0.0.1:46609}
   [junit4]   2> 483333 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [    ] o.e.j.s.Server Started @486105ms
   [junit4]   2> 483334 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest_D3F26B27978BDE60-001/control-001/cores, solr.data.dir=hdfs://localhost:50452/hdfs__localhost_50452__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-5.x_solr_build_solr-core_test_J0_temp_solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest_D3F26B27978BDE60-001_tempDir-002_control_data, hostPort=46609, hostContext=/}
   [junit4]   2> 483334 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@7b3cb2c6
   [junit4]   2> 483334 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest_D3F26B27978BDE60-001/control-001'
   [junit4]   2> 483334 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 483334 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 483335 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 483335 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 483337 INFO  (zkCallback-511-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@13112a0b name:ZooKeeperConnection Watcher:127.0.0.1:41962/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 483337 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 483337 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 483338 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 483338 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest_D3F26B27978BDE60-001/control-001/solr.xml
   [junit4]   2> 483356 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest_D3F26B27978BDE60-001/control-001/cores
   [junit4]   2> 483356 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [    ] o.a.s.c.CoreContainer New CoreContainer 263427016
   [junit4]   2> 483357 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest_D3F26B27978BDE60-001/control-001]
   [junit4]   2> 483357 WARN  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest_D3F26B27978BDE60-001/control-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest_D3F26B27978BDE60-001/control-001/lib
   [junit4]   2> 483357 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 483358 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 483359 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 483359 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 483359 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:41962/solr
   [junit4]   2> 483359 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 483360 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 483360 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 483362 INFO  (zkCallback-514-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@27072035 name:ZooKeeperConnection Watcher:127.0.0.1:41962 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 483362 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 483362 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 483364 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [n:127.0.0.1:46609_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 483365 INFO  (zkCallback-515-thread-1-processing-n:127.0.0.1:46609_) [n:127.0.0.1:46609_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@19029ed3 name:ZooKeeperConnection Watcher:127.0.0.1:41962/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 483365 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [n:127.0.0.1:46609_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 483367 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [n:127.0.0.1:46609_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 483369 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [n:127.0.0.1:46609_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 483372 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [n:127.0.0.1:46609_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 483374 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [n:127.0.0.1:46609_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 483376 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [n:127.0.0.1:46609_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 483379 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [n:127.0.0.1:46609_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 483380 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [n:127.0.0.1:46609_    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 483382 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [n:127.0.0.1:46609_    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 483383 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [n:127.0.0.1:46609_    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 483384 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [n:127.0.0.1:46609_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:46609_
   [junit4]   2> 483385 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [n:127.0.0.1:46609_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:46609_
   [junit4]   2> 483386 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [n:127.0.0.1:46609_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 483388 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [n:127.0.0.1:46609_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 483389 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [n:127.0.0.1:46609_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 483390 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [n:127.0.0.1:46609_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/95358870314483716-127.0.0.1:46609_-n_0000000000
   [junit4]   2> 483390 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [n:127.0.0.1:46609_    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:46609_
   [junit4]   2> 483391 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [n:127.0.0.1:46609_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 483392 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [n:127.0.0.1:46609_    ] o.a.s.c.Overseer Overseer (id=95358870314483716-127.0.0.1:46609_-n_0000000000) starting
   [junit4]   2> 483394 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [n:127.0.0.1:46609_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 483401 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [n:127.0.0.1:46609_    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 483401 INFO  (OverseerCollectionConfigSetProcessor-95358870314483716-127.0.0.1:46609_-n_0000000000) [n:127.0.0.1:46609_    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 483401 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [n:127.0.0.1:46609_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 483402 INFO  (OverseerStateUpdate-95358870314483716-127.0.0.1:46609_-n_0000000000) [n:127.0.0.1:46609_    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 483422 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [n:127.0.0.1:46609_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 483422 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [n:127.0.0.1:46609_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 483423 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [n:127.0.0.1:46609_    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest_D3F26B27978BDE60-001/control-001/cores
   [junit4]   2> 483424 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [n:127.0.0.1:46609_    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {config=solrconfig.xml, loadOnStartup=true, dataDir=data/, collection=control_collection, coreNodeName=, transient=false, shard=, configSetProperties=configsetprops.json, schema=schema.xml, name=collection1}
   [junit4]   2> 483424 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [n:127.0.0.1:46609_    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest_D3F26B27978BDE60-001/control-001/cores/collection1
   [junit4]   2> 483424 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [n:127.0.0.1:46609_    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 483425 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [n:127.0.0.1:46609_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0
   [junit4]   2> 483425 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [n:127.0.0.1:46609_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 483426 INFO  (coreLoadExecutor-1336-thread-1-processing-n:127.0.0.1:46609_) [n:127.0.0.1:46609_ c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 483426 INFO  (coreLoadExecutor-1336-thread-1-processing-n:127.0.0.1:46609_) [n:127.0.0.1:46609_ c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 483427 INFO  (OverseerStateUpdate-95358870314483716-127.0.0.1:46609_-n_0000000000) [n:127.0.0.1:46609_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "roles":null,
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:46609",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "node_name":"127.0.0.1:46609_",
   [junit4]   2>   "state":"down"} current state version: 0
   [junit4]   2> 483428 INFO  (OverseerStateUpdate-95358870314483716-127.0.0.1:46609_-n_0000000000) [n:127.0.0.1:46609_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "roles":null,
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:46609",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "node_name":"127.0.0.1:46609_",
   [junit4]   2>   "state":"down"}
   [junit4]   2> 483428 INFO  (coreLoadExecutor-1336-thread-1-processing-n:127.0.0.1:46609_) [n:127.0.0.1:46609_ c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 483428 INFO  (OverseerStateUpdate-95358870314483716-127.0.0.1:46609_-n_0000000000) [n:127.0.0.1:46609_    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 483429 INFO  (OverseerStateUpdate-95358870314483716-127.0.0.1:46609_-n_0000000000) [n:127.0.0.1:46609_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 483430 INFO  (zkCallback-515-thread-1-processing-n:127.0.0.1:46609_) [n:127.0.0.1:46609_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 484428 INFO  (coreLoadExecutor-1336-thread-1-processing-n:127.0.0.1:46609_) [n:127.0.0.1:46609_ c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 484429 INFO  (coreLoadExecutor-1336-thread-1-processing-n:127.0.0.1:46609_) [n:127.0.0.1:46609_ c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 484429 INFO  (coreLoadExecutor-1336-thread-1-processing-n:127.0.0.1:46609_) [n:127.0.0.1:46609_ c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 484429 INFO  (coreLoadExecutor-1336-thread-1-processing-n:127.0.0.1:46609_) [n:127.0.0.1:46609_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/control_collection]
   [junit4]   2> 484430 INFO  (coreLoadExecutor-1336-thread-1-processing-n:127.0.0.1:46609_) [n:127.0.0.1:46609_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/control_collection] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 484430 INFO  (coreLoadExecutor-1336-thread-1-processing-n:127.0.0.1:46609_) [n:127.0.0.1:46609_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest_D3F26B27978BDE60-001/control-001/cores/collection1'
   [junit4]   2> 484430 INFO  (coreLoadExecutor-1336-thread-1-processing-n:127.0.0.1:46609_) [n:127.0.0.1:46609_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 484430 INFO  (coreLoadExecutor-1336-thread-1-processing-n:127.0.0.1:46609_) [n:127.0.0.1:46609_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 484432 INFO  (coreLoadExecutor-1336-thread-1-processing-n:127.0.0.1:46609_) [n:127.0.0.1:46609_ c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 484447 INFO  (coreLoadExecutor-1336-thread-1-processing-n:127.0.0.1:46609_) [n:127.0.0.1:46609_ c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 484462 INFO  (coreLoadExecutor-1336-thread-1-processing-n:127.0.0.1:46609_) [n:127.0.0.1:46609_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.5.0
   [junit4]   2> 484495 INFO  (coreLoadExecutor-1336-thread-1-processing-n:127.0.0.1:46609_) [n:127.0.0.1:46609_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 484505 INFO  (coreLoadExecutor-1336-thread-1-processing-n:127.0.0.1:46609_) [n:127.0.0.1:46609_ c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 484757 INFO  (coreLoadExecutor-1336-thread-1-processing-n:127.0.0.1:46609_) [n:127.0.0.1:46609_ c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 484759 INFO  (coreLoadExecutor-1336-thread-1-processing-n:127.0.0.1:46609_) [n:127.0.0.1:46609_ c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 484760 INFO  (coreLoadExecutor-1336-thread-1-processing-n:127.0.0.1:46609_) [n:127.0.0.1:46609_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 484763 INFO  (coreLoadExecutor-1336-thread-1-processing-n:127.0.0.1:46609_) [n:127.0.0.1:46609_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 484784 INFO  (coreLoadExecutor-1336-thread-1-processing-n:127.0.0.1:46609_) [n:127.0.0.1:46609_ c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0
   [junit4]   2> 484784 INFO  (coreLoadExecutor-1336-thread-1-processing-n:127.0.0.1:46609_) [n:127.0.0.1:46609_ c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 484784 INFO  (coreLoadExecutor-1336-thread-1-processing-n:127.0.0.1:46609_) [n:127.0.0.1:46609_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.HdfsDirectoryFactory
   [junit4]   2> 484797 INFO  (coreLoadExecutor-1336-thread-1-processing-n:127.0.0.1:46609_) [n:127.0.0.1:46609_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:50452/solr_hdfs_home
   [junit4]   2> 484797 INFO  (coreLoadExecutor-1336-thread-1-processing-n:127.0.0.1:46609_) [n:127.0.0.1:46609_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
   [junit4]   2> 484798 INFO  (coreLoadExecutor-1336-thread-1-processing-n:127.0.0.1:46609_) [n:127.0.0.1:46609_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest_D3F26B27978BDE60-001/control-001/cores/collection1], dataDir=[hdfs://localhost:50452/solr_hdfs_home/control_collection/core_node1/data/]
   [junit4]   2> 484798 INFO  (coreLoadExecutor-1336-thread-1-processing-n:127.0.0.1:46609_) [n:127.0.0.1:46609_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@2302b6d2
   [junit4]   2> 484798 INFO  (coreLoadExecutor-1336-thread-1-processing-n:127.0.0.1:46609_) [n:127.0.0.1:46609_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:50452/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 484848 WARN  (coreLoadExecutor-1336-thread-1-processing-n:127.0.0.1:46609_) [n:127.0.0.1:46609_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.h.HdfsDirectory The NameNode is in SafeMode - Solr will wait 5 seconds and try again.
   [junit4]   2> 489851 WARN  (coreLoadExecutor-1336-thread-1-processing-n:127.0.0.1:46609_) [n:127.0.0.1:46609_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.h.HdfsDirectory The NameNode is in SafeMode - Solr will wait 5 seconds and try again.
   [junit4]   2> 494928 INFO  (coreLoadExecutor-1336-thread-1-processing-n:127.0.0.1:46609_) [n:127.0.0.1:46609_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:50452/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 494934 INFO  (coreLoadExecutor-1336-thread-1-processing-n:127.0.0.1:46609_) [n:127.0.0.1:46609_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=hdfs://localhost:50452/solr_hdfs_home/control_collection/core_node1/data/index/
   [junit4]   2> 494965 WARN  (coreLoadExecutor-1336-thread-1-processing-n:127.0.0.1:46609_) [n:127.0.0.1:46609_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory 'hdfs:/localhost:50452/solr_hdfs_home/control_collection/core_node1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 494965 INFO  (coreLoadExecutor-1336-thread-1-processing-n:127.0.0.1:46609_) [n:127.0.0.1:46609_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:50452/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 494977 INFO  (coreLoadExecutor-1336-thread-1-processing-n:127.0.0.1:46609_) [n:127.0.0.1:46609_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
   [junit4]   2> 494977 INFO  (coreLoadExecutor-1336-thread-1-processing-n:127.0.0.1:46609_) [n:127.0.0.1:46609_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
   [junit4]   2> 494977 INFO  (coreLoadExecutor-1336-thread-1-processing-n:127.0.0.1:46609_) [n:127.0.0.1:46609_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Creating new global HDFS BlockCache
   [junit4]   2> 495014 INFO  (coreLoadExecutor-1336-thread-1-processing-n:127.0.0.1:46609_) [n:127.0.0.1:46609_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
   [junit4]   2> 495014 INFO  (coreLoadExecutor-1336-thread-1-processing-n:127.0.0.1:46609_) [n:127.0.0.1:46609_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:50452/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 495015 INFO  (coreLoadExecutor-1336-thread-1-processing-n:127.0.0.1:46609_) [n:127.0.0.1:46609_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@271f38eb
   [junit4]   2> 495347 INFO  (IPC Server handler 8 on 50452) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:56507 is added to blk_1073741825_1001{blockUCState=COMMITTED, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-4da58444-b95f-4e3f-9a63-2cf8ccd2332f:NORMAL:127.0.0.1:56507|RBW], ReplicaUnderConstruction[[DISK]DS-6c33d942-3867-45c1-b1aa-26001407115b:NORMAL:127.0.0.1:47070|RBW]]} size 71
   [junit4]   2> 495347 INFO  (IPC Server handler 4 on 50452) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:47070 is added to blk_1073741825_1001 size 71
   [junit4]   2> 495754 INFO  (coreLoadExecutor-1336-thread-1-processing-n:127.0.0.1:46609_) [n:127.0.0.1:46609_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(BlockDirectory(HdfsDirectory@hdfs://localhost:50452/solr_hdfs_home/control_collection/core_node1/data/index lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1f87824f); maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 495754 INFO  (coreLoadExecutor-1336-thread-1-processing-n:127.0.0.1:46609_) [n:127.0.0.1:46609_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 495757 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:46609_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in hdfs://localhost:50452/solr_hdfs_home/control_collection/core_node1/data/
   [junit4]   2> 495760 INFO  (coreLoadExecutor-1336-thread-1-processing-n:127.0.0.1:46609_) [n:127.0.0.1:46609_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 495761 INFO  (coreLoadExecutor-1336-thread-1-processing-n:127.0.0.1:46609_) [n:127.0.0.1:46609_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 495761 INFO  (coreLoadExecutor-1336-thread-1-processing-n:127.0.0.1:46609_) [n:127.0.0.1:46609_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 495761 INFO  (coreLoadExecutor-1336-thread-1-processing-n:127.0.0.1:46609_) [n:127.0.0.1:46609_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 495761 INFO  (coreLoadExecutor-1336-thread-1-processing-n:127.0.0.1:46609_) [n:127.0.0.1:46609_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 495761 INFO  (coreLoadExecutor-1336-thread-1-processing-n:127.0.0.1:46609_) [n:127.0.0.1:46609_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 495761 INFO  (coreLoadExecutor-1336-thread-1-processing-n:127.0.0.1:46609_) [n:127.0.0.1:46609_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 495762 INFO  (coreLoadExecutor-1336-thread-1-processing-n:127.0.0.1:46609_) [n:127.0.0.1:46609_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 495762 INFO  (coreLoadExecutor-1336-thread-1-processing-n:127.0.0.1:46609_) [n:127.0.0.1:46609_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 495762 WARN  (coreLoadExecutor-1336-thread-1-processing-n:127.0.0.1:46609_) [n:127.0.0.1:46609_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
   [junit4]   2> 495765 INFO  (coreLoadExecutor-1336-thread-1-processing-n:127.0.0.1:46609_) [n:127.0.0.1:46609_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 495766 INFO  (coreLoadExecutor-1336-thread-1-processing-n:127.0.0.1:46609_) [n:127.0.0.1:46609_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 495768 INFO  (coreLoadExecutor-1336-thread-1-processing-n:127.0.0.1:46609_) [n:127.0.0.1:46609_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 495769 INFO  (coreLoadExecutor-1336-thread-1-processing-n:127.0.0.1:46609_) [n:127.0.0.1:46609_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 495773 INFO  (coreLoadExecutor-1336-thread-1-processing-n:127.0.0.1:46609_) [n:127.0.0.1:46609_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /dump,/admin/properties,/replication,/admin/file,/admin/system,/admin/logging,standard,/schema,/admin/luke,/config,/get,/admin/plugins,/update/csv,/update/json/docs,/update,/admin/threads,/admin/mbeans,/update/json,/admin/segments,/admin/ping
   [junit4]   2> 495775 INFO  (coreLoadExecutor-1336-thread-1-processing-n:127.0.0.1:46609_) [n:127.0.0.1:46609_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 495780 INFO  (coreLoadExecutor-1336-thread-1-processing-n:127.0.0.1:46609_) [n:127.0.0.1:46609_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
   [junit4]   2> 495780 INFO  (coreLoadExecutor-1336-thread-1-processing-n:127.0.0.1:46609_) [n:127.0.0.1:46609_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 tlogDfsReplication=2
   [junit4]   2> 495798 INFO  (coreLoadExecutor-1336-thread-1-processing-n:127.0.0.1:46609_) [n:127.0.0.1:46609_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: if uncommited for 15000ms; 
   [junit4]   2> 495798 INFO  (coreLoadExecutor-1336-thread-1-processing-n:127.0.0.1:46609_) [n:127.0.0.1:46609_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 495801 INFO  (coreLoadExecutor-1336-thread-1-processing-n:127.0.0.1:46609_) [n:127.0.0.1:46609_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=11, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.44944911720981784]
   [junit4]   2> 495885 INFO  (coreLoadExecutor-1336-thread-1-processing-n:127.0.0.1:46609_) [n:127.0.0.1:46609_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(BlockDirectory(HdfsDirectory@hdfs://localhost:50452/solr_hdfs_home/control_collection/core_node1/data/index lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1f87824f); maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 495885 INFO  (coreLoadExecutor-1336-thread-1-processing-n:127.0.0.1:46609_) [n:127.0.0.1:46609_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 495885 INFO  (coreLoadExecutor-1336-thread-1-processing-n:127.0.0.1:46609_) [n:127.0.0.1:46609_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@2ecdc5c4[collection1] main
   [junit4]   2> 495886 INFO  (coreLoadExecutor-1336-thread-1-processing-n:127.0.0.1:46609_) [n:127.0.0.1:46609_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/control_collection]
   [junit4]   2> 495888 INFO  (coreLoadExecutor-1336-thread-1-processing-n:127.0.0.1:46609_) [n:127.0.0.1:46609_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/control_collection] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 495888 INFO  (coreLoadExecutor-1336-thread-1-processing-n:127.0.0.1:46609_) [n:127.0.0.1:46609_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 495892 INFO  (coreLoadExecutor-1336-thread-1-processing-n:127.0.0.1:46609_) [n:127.0.0.1:46609_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 495892 INFO  (coreLoadExecutor-1336-thread-1-processing-n:127.0.0.1:46609_) [n:127.0.0.1:46609_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 495892 INFO  (coreLoadExecutor-1336-thread-1-processing-n:127.0.0.1:46609_) [n:127.0.0.1:46609_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 495893 INFO  (coreLoadExecutor-1336-thread-1-processing-n:127.0.0.1:46609_) [n:127.0.0.1:46609_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 495893 INFO  (coreLoadExecutor-1336-thread-1-processing-n:127.0.0.1:46609_) [n:127.0.0.1:46609_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 495893 INFO  (coreLoadExecutor-1336-thread-1-processing-n:127.0.0.1:46609_) [n:127.0.0.1:46609_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 495896 INFO  (coreLoadExecutor-1336-thread-1-processing-n:127.0.0.1:46609_) [n:127.0.0.1:46609_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 495898 INFO  (searcherExecutor-1337-thread-1-processing-r:core_node1 s:shard1 c:control_collection x:collection1 n:127.0.0.1:46609_) [n:127.0.0.1:46609_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@2ecdc5c4[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 495898 INFO  (coreLoadExecutor-1336-thread-1-processing-n:127.0.0.1:46609_) [n:127.0.0.1:46609_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 495898 INFO  (coreLoadExecutor-1336-thread-1-processing-n:127.0.0.1:46609_) [n:127.0.0.1:46609_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 256 version buckets from index
   [junit4]   2> 495899 INFO  (coreLoadExecutor-1336-thread-1-processing-n:127.0.0.1:46609_) [n:127.0.0.1:46609_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 495899 INFO  (coreLoadExecutor-1336-thread-1-processing-n:127.0.0.1:46609_) [n:127.0.0.1:46609_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1525741938479726592
   [junit4]   2> 495899 INFO  (coreLoadExecutor-1336-thread-1-processing-n:127.0.0.1:46609_) [n:127.0.0.1:46609_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 0.0ms to seed version buckets with highest version 1525741938479726592
   [junit4]   2> 495899 INFO  (coreLoadExecutor-1336-thread-1-processing-n:127.0.0.1:46609_) [n:127.0.0.1:46609_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 495899 INFO  (coreLoadExecutor-1336-thread-1-processing-n:127.0.0.1:46609_) [n:127.0.0.1:46609_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 495900 INFO  (coreZkRegister-1330-thread-1-processing-r:core_node1 s:shard1 c:control_collection x:collection1 n:127.0.0.1:46609_) [n:127.0.0.1:46609_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:46609 collection:control_collection shard:shard1
   [junit4]   2> 495901 INFO  (coreZkRegister-1330-thread-1-processing-r:core_node1 s:shard1 c:control_collection x:collection1 n:127.0.0.1:46609_) [n:127.0.0.1:46609_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 495905 INFO  (coreZkRegister-1330-thread-1-processing-r:core_node1 s:shard1 c:control_collection x:collection1 n:127.0.0.1:46609_) [n:127.0.0.1:46609_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/95358870314483716-core_node1-n_0000000000
   [junit4]   2> 495905 INFO  (coreZkRegister-1330-thread-1-processing-r:core_node1 s:shard1 c:control_collection x:collection1 n:127.0.0.1:46609_) [n:127.0.0.1:46609_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard=shard1 and weAreReplacement=false and leaderVoteWait=10000
   [junit4]   2> 495907 INFO  (coreZkRegister-1330-thread-1-processing-r:core_node1 s:shard1 c:control_collection x:collection1 n:127.0.0.1:46609_) [n:127.0.0.1:46609_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 495907 INFO  (coreZkRegister-1330-thread-1-processing-r:core_node1 s:shard1 c:control_collection x:collection1 n:127.0.0.1:46609_) [n:127.0.0.1:46609_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 495907 INFO  (coreZkRegister-1330-thread-1-processing-r:core_node1 s:shard1 c:control_collection x:collection1 n:127.0.0.1:46609_) [n:127.0.0.1:46609_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:46609/collection1/
   [junit4]   2> 495907 INFO  (coreZkRegister-1330-thread-1-processing-r:core_node1 s:shard1 c:control_collection x:collection1 n:127.0.0.1:46609_) [n:127.0.0.1:46609_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 495907 INFO  (coreZkRegister-1330-thread-1-processing-r:core_node1 s:shard1 c:control_collection x:collection1 n:127.0.0.1:46609_) [n:127.0.0.1:46609_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:46609/collection1/ has no replicas
   [junit4]   2> 495908 INFO  (OverseerStateUpdate-95358870314483716-127.0.0.1:46609_-n_0000000000) [n:127.0.0.1:46609_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 495909 INFO  (coreZkRegister-1330-thread-1-processing-r:core_node1 s:shard1 c:control_collection x:collection1 n:127.0.0.1:46609_) [n:127.0.0.1:46609_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 495909 INFO  (zkCallback-515-thread-1-processing-n:127.0.0.1:46609_) [n:127.0.0.1:46609_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 495913 INFO  (coreZkRegister-1330-thread-1-processing-r:core_node1 s:shard1 c:control_collection x:collection1 n:127.0.0.1:46609_) [n:127.0.0.1:46609_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 495915 INFO  (coreZkRegister-1330-thread-1-processing-r:core_node1 s:shard1 c:control_collection x:collection1 n:127.0.0.1:46609_) [n:127.0.0.1:46609_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:46609/collection1/ shard1
   [junit4]   2> 495916 INFO  (OverseerStateUpdate-95358870314483716-127.0.0.1:46609_-n_0000000000) [n:127.0.0.1:46609_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:46609",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 2
   [junit4]   2> 495930 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 495931 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 495932 INFO  (zkCallback-517-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2776f0c4 name:ZooKeeperConnection Watcher:127.0.0.1:41962/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 495932 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 495933 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 495933 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 495935 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 495935 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with stateFormat=2
   [junit4]   2> 495935 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 495936 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 495937 INFO  (zkCallback-518-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6767daac name:ZooKeeperConnection Watcher:127.0.0.1:41962/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 495937 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 495937 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 495940 INFO  (OverseerStateUpdate-95358870314483716-127.0.0.1:46609_-n_0000000000) [n:127.0.0.1:46609_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"create",
   [junit4]   2>   "name":"collection1",
   [junit4]   2>   "numShards":"4",
   [junit4]   2>   "stateFormat":"2"} current state version: 2
   [junit4]   2> 495940 INFO  (OverseerStateUpdate-95358870314483716-127.0.0.1:46609_-n_0000000000) [n:127.0.0.1:46609_    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 495944 INFO  (zkCallback-515-thread-1-processing-n:127.0.0.1:46609_) [n:127.0.0.1:46609_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 495944 INFO  (zkCallback-517-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 495966 INFO  (coreZkRegister-1330-thread-1-processing-r:core_node1 s:shard1 c:control_collection x:collection1 n:127.0.0.1:46609_) [n:127.0.0.1:46609_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:46609/collection1/ and leader is http://127.0.0.1:46609/collection1/
   [junit4]   2> 495966 INFO  (coreZkRegister-1330-thread-1-processing-r:core_node1 s:shard1 c:control_collection x:collection1 n:127.0.0.1:46609_) [n:127.0.0.1:46609_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:46609
   [junit4]   2> 495966 INFO  (coreZkRegister-1330-thread-1-processing-r:core_node1 s:shard1 c:control_collection x:collection1 n:127.0.0.1:46609_) [n:127.0.0.1:46609_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 495966 INFO  (coreZkRegister-1330-thread-1-processing-r:core_node1 s:shard1 c:control_collection x:collection1 n:127.0.0.1:46609_) [n:127.0.0.1:46609_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 495966 INFO  (coreZkRegister-1330-thread-1-processing-r:core_node1 s:shard1 c:control_collection x:collection1 n:127.0.0.1:46609_) [n:127.0.0.1:46609_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 495969 INFO  (OverseerStateUpdate-95358870314483716-127.0.0.1:46609_-n_0000000000) [n:127.0.0.1:46609_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "roles":null,
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:46609",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "numShards":"4",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "node_name":"127.0.0.1:46609_",
   [junit4]   2>   "state":"active"} current state version: 3
   [junit4]   2> 495969 INFO  (OverseerStateUpdate-95358870314483716-127.0.0.1:46609_-n_0000000000) [n:127.0.0.1:46609_    ] o.a.s.c.o.ReplicaMutator Update state numShards=4 message={
   [junit4]   2>   "roles":null,
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:46609",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "numShards":"4",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "node_name":"127.0.0.1:46609_",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 495970 INFO  (OverseerStateUpdate-95358870314483716-127.0.0.1:46609_-n_0000000000) [n:127.0.0.1:46609_    ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/collection1/state.json
   [junit4]   2> 496057 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest_D3F26B27978BDE60-001/shard-1-001/cores/collection1
   [junit4]   2> 496058 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest_D3F26B27978BDE60-001/shard-1-001
   [junit4]   2> 496059 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 496062 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5f1459a1{/,null,AVAILABLE}
   [junit4]   2> 496062 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [    ] o.e.j.s.ServerConnector Started ServerConnector@278e8793{HTTP/1.1}{127.0.0.1:42685}
   [junit4]   2> 496062 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [    ] o.e.j.s.Server Started @498834ms
   [junit4]   2> 496062 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/, hostPort=42685, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest_D3F26B27978BDE60-001/shard-1-001/cores, solr.data.dir=hdfs://localhost:50452/hdfs__localhost_50452__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-5.x_solr_build_solr-core_test_J0_temp_solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest_D3F26B27978BDE60-001_tempDir-002_jetty1, solrconfig=solrconfig.xml}
   [junit4]   2> 496063 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@7b3cb2c6
   [junit4]   2> 496063 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest_D3F26B27978BDE60-001/shard-1-001'
   [junit4]   2> 496063 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 496063 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 496063 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 496065 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 496067 INFO  (zkCallback-519-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@59308938 name:ZooKeeperConnection Watcher:127.0.0.1:41962/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 496067 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 496067 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 496072 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 496072 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest_D3F26B27978BDE60-001/shard-1-001/solr.xml
   [junit4]   2> 496075 INFO  (zkCallback-517-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 496075 INFO  (zkCallback-515-thread-1-processing-n:127.0.0.1:46609_) [n:127.0.0.1:46609_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 496088 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest_D3F26B27978BDE60-001/shard-1-001/cores
   [junit4]   2> 496088 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [    ] o.a.s.c.CoreContainer New CoreContainer 1661435035
   [junit4]   2> 496088 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest_D3F26B27978BDE60-001/shard-1-001]
   [junit4]   2> 496088 WARN  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest_D3F26B27978BDE60-001/shard-1-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest_D3F26B27978BDE60-001/shard-1-001/lib
   [junit4]   2> 496089 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 496090 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 496091 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 496091 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 496091 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:41962/solr
   [junit4]   2> 496091 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 496091 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 496091 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 496093 INFO  (zkCallback-522-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@36595769 name:ZooKeeperConnection Watcher:127.0.0.1:41962 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 496093 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 496093 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 496095 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [n:127.0.0.1:42685_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 496096 INFO  (zkCallback-523-thread-1-processing-n:127.0.0.1:42685_) [n:127.0.0.1:42685_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5ef1e6e4 name:ZooKeeperConnection Watcher:127.0.0.1:41962/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 496096 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [n:127.0.0.1:42685_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 496102 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [n:127.0.0.1:42685_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 497109 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [n:127.0.0.1:42685_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:42685_
   [junit4]   2> 497109 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [n:127.0.0.1:42685_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:42685_
   [junit4]   2> 497111 INFO  (zkCallback-517-thread-1) [    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 497111 INFO  (zkCallback-523-thread-1-processing-n:127.0.0.1:42685_) [n:127.0.0.1:42685_    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 497111 INFO  (zkCallback-515-thread-1-processing-n:127.0.0.1:46609_) [n:127.0.0.1:46609_    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 497113 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [n:127.0.0.1:42685_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 497113 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [n:127.0.0.1:42685_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/95358870314483721-127.0.0.1:42685_-n_0000000001
   [junit4]   2> 497114 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [n:127.0.0.1:42685_    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/95358870314483716-127.0.0.1:46609_-n_0000000000 to know if I could be the leader
   [junit4]   2> 497139 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [n:127.0.0.1:42685_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 497139 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [n:127.0.0.1:42685_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 497140 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [n:127.0.0.1:42685_    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest_D3F26B27978BDE60-001/shard-1-001/cores
   [junit4]   2> 497141 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [n:127.0.0.1:42685_    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {coreNodeName=, loadOnStartup=true, configSetProperties=configsetprops.json, name=collection1, collection=collection1, schema=schema.xml, config=solrconfig.xml, shard=, transient=false, dataDir=data/}
   [junit4]   2> 497141 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [n:127.0.0.1:42685_    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest_D3F26B27978BDE60-001/shard-1-001/cores/collection1
   [junit4]   2> 497141 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [n:127.0.0.1:42685_    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 497143 INFO  (coreLoadExecutor-1347-thread-1-processing-n:127.0.0.1:42685_) [n:127.0.0.1:42685_ c:collection1   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 497143 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [n:127.0.0.1:42685_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0
   [junit4]   2> 497143 INFO  (coreLoadExecutor-1347-thread-1-processing-n:127.0.0.1:42685_) [n:127.0.0.1:42685_ c:collection1   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 497143 INFO  (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[D3F26B27978BDE60]) [n:127.0.0.1:42685_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 497145 INFO  (coreLoadExecutor-1347-thread-1-processing-n:127.0.0.1:42685_) [n:127.0.0.1:42685_ c:collection1   x:collection1] o.a.s.c.ZkController Registering watch for collection collection1
   [junit4]   2> 497145 INFO  (coreLoadExecutor-1347-thread-1-processing-n:127.0.0.1:42685_) [n:127.0.0.1:42685_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader addZkWatch [collection1]
   [junit4]   2> 497145 INFO  (OverseerStateUpdate-95358870314483716-127.0.0.1:46609_-n_0000000000) [n:127.0.0.1:46609_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "roles":null,
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "base_url":"http://127.0.0.1:42685",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "numShards":"4",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "node_name":"127.0.0.1:42685_",
   [junit4]   2>   "state":"down"} current state version: 4
   [junit4]   2> 497145 INFO  (coreLoadExecutor-1347-thread-1-processing-n:127.0.0.1:42685_) [n:127.0.0.1:42685_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Add data for [collection1] ver [0]
   [junit4]   2> 497145 INFO  (OverseerStateUpdate-95358870314483716-127.0.0.1:46609_-n_0000000000) [n:127.0.0.1:46609_    ] o.a.s.c.o.ReplicaMutator Update state numShards=4 message={
   [junit4]   2>   "roles":null,
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "base_url":"http://127.0.0.1:42685",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "numShards":"4",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "node_name":"127.0.0.1:42685_",
   [junit4]   2>   "state":"down"}
   [junit4]   2> 497145 INFO  (coreLoadExecutor-1347-thread-1-processing-n:127.0.0.1:42685_) [n:127.0.0.1:42685_ c:collection1   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 497145 INFO  (OverseerStateUpdate-95358870314483716-127.0.0.1:46609_-n_0000000000) [n:127.0.0.1:46609_    ] o.a.s.c.o.ReplicaMutator Collection already exists with numShards=4
   [junit4]   2> 497146 INFO  (OverseerStateUpdate-95358870314483716-127.0.0.1:46609_-n_0000000000) [n:127.0.0.1:46609_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 497248 INFO  (OverseerStateUpdate-95358870314483716-127.0.0.1:46609_-n_0000000000) [n:127.0.0.1:46609_    ] o.a.s.c.o.ZkStateWriter going to update_collection /collections/collection1/state.json version: 0
   [junit4]   2> 497249 INFO  (zkCallback-523-thread-1-processing-n:127.0.0.1:42685_) [n:127.0.0.1:42685_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 497250 INFO  (zkCallback-523-thread-1-processing-n:127.0.0.1:42685_) [n:127.0.0.1:42685_    ] o.a.s.c.c.ZkStateReader Updating data for [collection1] from [0] to [1]
   [junit4]   2> 498146 INFO  (coreLoadExecutor-1347-thread-1-processing-n:127.0.0.1:42685_) [n:127.0.0.1:42685_ c:collection1   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 498146 INFO  (coreLoadExecutor-1347-thread-1-processing-n:127.0.0.1:42685_) [n:127.0.0.1:42685_ c:collection1   x:collection1] o.a.s.c.ZkController Check for collection zkNode:collection1
   [junit4]   2> 498146 INFO  (coreLoadExecutor-1347-thread-1-processing-n:127.0.0.1:42685_) [n:127.0.0.1:42685_ c:collection1   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 498146 INFO  (coreLoadExecutor-1347-thread-1-processing-n:127.0.0.1:42685_) [n:127.0.0.1:42685_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/collection1]
   [junit4]   2> 498147 INFO  (coreLoadExecutor-1347-thread-1-processing-n:127.0.0.1:42685_) [n:127.0.0.1:42685_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/collection1] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 498147 INFO  (coreLoadExecutor-1347-thread-1-processing-n:127.0.0.1:42685_) [n:127.0.0.1:42685_ c:collection1   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest_D3F26B27978BDE60-001/shard-1-001/cores/collection1'
   [junit4]   2> 498147 INFO  (coreLoadExecutor-1347-thread-1-processing-n:127.0.0.1:42685_) [n:127.0.0.1:42685_ c:collection1   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 498147 INFO  (coreLoadExecutor-1347-thread-1-processing-n:127.0.0.1:42685_) [n:127.0.0.1:42685_ c:collection1   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 498151 INFO  (coreLoadExecutor-1347-thread-1-processing-n:127.0.0.1:42685_) [n:127.0.0.1:42685_ c:collection1   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 498161 INFO  (coreLoadExecutor-1347-thread-1-

[...truncated too long message...]

rs-admin-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-authn-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-authz-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-changelog-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-collective-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-event-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-exception-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-journal-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-normalization-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-operational-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-referral-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-schema-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-subtree-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-trigger-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-jdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-jdbm1-2.0.0-M2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-kerberos-codec-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-ldif-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-mavibot-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-protocol-kerberos-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-protocol-ldap-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-protocol-shared-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-xdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/api-all-1.0.0-M20.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/bcprov-jdk15-1.45.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/cglib-nodep-2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/commons-collections-3.2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/commons-math3-3.4.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/easymock-3.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/ehcache-core-2.4.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/hadoop-common-2.6.0-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/hadoop-hdfs-2.6.0-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/hadoop-minikdc-2.6.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jackson-annotations-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jackson-databind-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jersey-core-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jersey-server-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jetty-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jetty-util-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/mina-core-2.0.0-M5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/objenesis-1.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/analysis/icu/lucene-analyzers-icu-5.5.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-analysis-extras/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/queryparser/classes/test:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/contrib/analysis-extras/lib/icu4j-54.1.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-launcher.jar:/x1/jenkins/.ant/lib/ivy-2.3.0.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-antlr.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-regexp.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-resolver.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-net.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jmf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-oro.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-xalan2.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-logging.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jsch.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit4.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-javamail.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jai.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jdepend.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bsf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-swing.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-netrexx.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bcel.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-testutil.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-log4j.jar:/x1/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.7/lib/tools.jar:/x1/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.3.2.jar com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J1-20160209_232641_349.events @/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J1-20160209_232641_349.suites -stdin
   [junit4] ERROR: JVM J1 ended with an exception: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1501)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:121)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:955)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:952)
   [junit4] 	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4] 	at java.lang.Thread.run(Thread.java:745)

BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:757: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:694: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:534: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1477: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1033: At least one slave process threw an exception, first: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.

Total time: 230 minutes 22 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1092 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1092/

1 tests failed.
FAILED:  org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.test

Error Message:
Timeout waiting for all live and active

Stack Trace:
java.lang.AssertionError: Timeout waiting for all live and active
	at __randomizedtesting.SeedInfo.seed([C2DBDF2397FC8F2D:4A8FE0F93900E2D5]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.testBasics(SharedFSAutoReplicaFailoverTest.java:193)
	at org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.test(SharedFSAutoReplicaFailoverTest.java:107)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:964)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:939)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 11207 lines...]
   [junit4] Suite: org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_C2DBDF2397FC8F2D-001/init-core-data-001
   [junit4]   2> 61103 INFO  (SUITE-SharedFSAutoReplicaFailoverTest-seed#[C2DBDF2397FC8F2D]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   2> 61972 WARN  (SUITE-SharedFSAutoReplicaFailoverTest-seed#[C2DBDF2397FC8F2D]-worker) [    ] o.a.h.u.NativeCodeLoader Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
   [junit4]   1> Formatting using clusterid: testClusterID
   [junit4]   2> 62962 WARN  (SUITE-SharedFSAutoReplicaFailoverTest-seed#[C2DBDF2397FC8F2D]-worker) [    ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
   [junit4]   2> 63185 INFO  (SUITE-SharedFSAutoReplicaFailoverTest-seed#[C2DBDF2397FC8F2D]-worker) [    ] o.m.log Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
   [junit4]   2> 63196 WARN  (SUITE-SharedFSAutoReplicaFailoverTest-seed#[C2DBDF2397FC8F2D]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 63291 INFO  (SUITE-SharedFSAutoReplicaFailoverTest-seed#[C2DBDF2397FC8F2D]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 63348 INFO  (SUITE-SharedFSAutoReplicaFailoverTest-seed#[C2DBDF2397FC8F2D]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/hdfs to ./temp/Jetty_localhost_37576_hdfs____.27w9ye/webapp
   [junit4]   2> 63573 INFO  (SUITE-SharedFSAutoReplicaFailoverTest-seed#[C2DBDF2397FC8F2D]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 64285 INFO  (SUITE-SharedFSAutoReplicaFailoverTest-seed#[C2DBDF2397FC8F2D]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:37576
   [junit4]   2> 65455 WARN  (SUITE-SharedFSAutoReplicaFailoverTest-seed#[C2DBDF2397FC8F2D]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 65463 INFO  (SUITE-SharedFSAutoReplicaFailoverTest-seed#[C2DBDF2397FC8F2D]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 65479 INFO  (SUITE-SharedFSAutoReplicaFailoverTest-seed#[C2DBDF2397FC8F2D]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_45619_datanode____.2joyxo/webapp
   [junit4]   2> 65630 INFO  (SUITE-SharedFSAutoReplicaFailoverTest-seed#[C2DBDF2397FC8F2D]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 65936 INFO  (SUITE-SharedFSAutoReplicaFailoverTest-seed#[C2DBDF2397FC8F2D]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:45619
   [junit4]   2> 66081 WARN  (SUITE-SharedFSAutoReplicaFailoverTest-seed#[C2DBDF2397FC8F2D]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 66084 INFO  (SUITE-SharedFSAutoReplicaFailoverTest-seed#[C2DBDF2397FC8F2D]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 66105 INFO  (SUITE-SharedFSAutoReplicaFailoverTest-seed#[C2DBDF2397FC8F2D]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_36447_datanode____.rl0145/webapp
   [junit4]   2> 66257 INFO  (SUITE-SharedFSAutoReplicaFailoverTest-seed#[C2DBDF2397FC8F2D]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 66569 INFO  (SUITE-SharedFSAutoReplicaFailoverTest-seed#[C2DBDF2397FC8F2D]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:36447
   [junit4]   2> 67196 INFO  (IPC Server handler 6 on 42030) [    ] BlockStateChange BLOCK* processReport: from storage DS-aaa45165-7661-4482-9510-794e8d6ca878 node DatanodeRegistration(127.0.0.1, datanodeUuid=94f935dc-684d-4978-ac50-555285163e1a, infoPort=45619, ipcPort=42467, storageInfo=lv=-56;cid=testClusterID;nsid=1065114925;c=0), blocks: 0, hasStaleStorages: true, processing time: 3 msecs
   [junit4]   2> 67196 INFO  (IPC Server handler 7 on 42030) [    ] BlockStateChange BLOCK* processReport: from storage DS-e1deea49-aafd-4081-892d-e6b43ad92332 node DatanodeRegistration(127.0.0.1, datanodeUuid=f5255e05-99eb-46a6-9953-974009fbd2d0, infoPort=36447, ipcPort=38502, storageInfo=lv=-56;cid=testClusterID;nsid=1065114925;c=0), blocks: 0, hasStaleStorages: true, processing time: 1 msecs
   [junit4]   2> 67197 INFO  (IPC Server handler 6 on 42030) [    ] BlockStateChange BLOCK* processReport: from storage DS-290dd1ea-1900-4c79-8fcb-289d117fd3aa node DatanodeRegistration(127.0.0.1, datanodeUuid=94f935dc-684d-4978-ac50-555285163e1a, infoPort=45619, ipcPort=42467, storageInfo=lv=-56;cid=testClusterID;nsid=1065114925;c=0), blocks: 0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 67197 INFO  (IPC Server handler 7 on 42030) [    ] BlockStateChange BLOCK* processReport: from storage DS-ce3f19da-4b31-4e5f-8db9-0034100d9215 node DatanodeRegistration(127.0.0.1, datanodeUuid=f5255e05-99eb-46a6-9953-974009fbd2d0, infoPort=36447, ipcPort=38502, storageInfo=lv=-56;cid=testClusterID;nsid=1065114925;c=0), blocks: 0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 67533 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 67534 INFO  (Thread-256) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 67534 INFO  (Thread-256) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 67633 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [    ] o.a.s.c.ZkTestServer start zk server on port:43763
   [junit4]   2> 67634 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 67634 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 67638 INFO  (zkCallback-65-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6db66fb4 name:ZooKeeperConnection Watcher:127.0.0.1:43763 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 67638 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 67638 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 67639 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 67642 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 67643 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 67644 INFO  (zkCallback-66-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@118255e5 name:ZooKeeperConnection Watcher:127.0.0.1:43763/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 67645 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 67645 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 67645 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 67648 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 67651 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 67653 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 67655 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 67655 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 67660 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 67660 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 67663 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 67663 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 67667 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 67667 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 67670 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 67670 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 67673 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 67673 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 67676 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 67676 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 67679 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 67679 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 67681 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 67682 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 67684 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 67685 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 67687 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 67687 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 67805 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_C2DBDF2397FC8F2D-001/control-001/cores/collection1
   [junit4]   2> 67807 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 67810 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@6dfd6e00{/,null,AVAILABLE}
   [junit4]   2> 67811 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [    ] o.e.j.s.ServerConnector Started ServerConnector@78f85219{HTTP/1.1}{127.0.0.1:53112}
   [junit4]   2> 67811 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [    ] o.e.j.s.Server Started @70942ms
   [junit4]   2> 67811 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_C2DBDF2397FC8F2D-001/control-001/cores, hostPort=53112}
   [junit4]   2> 67812 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@7b3cb2c6
   [junit4]   2> 67812 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_C2DBDF2397FC8F2D-001/control-001'
   [junit4]   2> 67812 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 67812 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 67813 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 67814 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 67818 INFO  (zkCallback-67-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@442e80f6 name:ZooKeeperConnection Watcher:127.0.0.1:43763/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 67819 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 67819 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 67821 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 67821 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_C2DBDF2397FC8F2D-001/control-001/solr.xml
   [junit4]   2> 67858 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_C2DBDF2397FC8F2D-001/control-001/cores
   [junit4]   2> 67858 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [    ] o.a.s.c.CoreContainer New CoreContainer 1594567021
   [junit4]   2> 67858 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_C2DBDF2397FC8F2D-001/control-001]
   [junit4]   2> 67859 WARN  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_C2DBDF2397FC8F2D-001/control-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_C2DBDF2397FC8F2D-001/control-001/lib
   [junit4]   2> 67859 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 67860 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 67861 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 67861 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 67862 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:43763/solr
   [junit4]   2> 67862 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 67862 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 67863 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 67866 INFO  (zkCallback-70-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@27a020d6 name:ZooKeeperConnection Watcher:127.0.0.1:43763 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 67866 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 67867 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 67869 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [n:127.0.0.1:53112_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 67871 INFO  (zkCallback-71-thread-1-processing-n:127.0.0.1:53112_) [n:127.0.0.1:53112_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@213ad2f0 name:ZooKeeperConnection Watcher:127.0.0.1:43763/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 67871 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [n:127.0.0.1:53112_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 67873 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [n:127.0.0.1:53112_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 67876 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [n:127.0.0.1:53112_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 67881 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [n:127.0.0.1:53112_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 67886 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [n:127.0.0.1:53112_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 67889 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [n:127.0.0.1:53112_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 67892 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [n:127.0.0.1:53112_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 67894 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [n:127.0.0.1:53112_    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 67896 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [n:127.0.0.1:53112_    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 67898 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [n:127.0.0.1:53112_    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 67901 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [n:127.0.0.1:53112_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:53112_
   [junit4]   2> 67903 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [n:127.0.0.1:53112_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:53112_
   [junit4]   2> 67907 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [n:127.0.0.1:53112_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 67909 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [n:127.0.0.1:53112_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 67911 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [n:127.0.0.1:53112_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 67912 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [n:127.0.0.1:53112_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/95353104697655300-127.0.0.1:53112_-n_0000000000
   [junit4]   2> 67912 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [n:127.0.0.1:53112_    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:53112_
   [junit4]   2> 67912 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [n:127.0.0.1:53112_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 67914 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [n:127.0.0.1:53112_    ] o.a.s.c.Overseer Overseer (id=95353104697655300-127.0.0.1:53112_-n_0000000000) starting
   [junit4]   2> 67917 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [n:127.0.0.1:53112_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 67926 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [n:127.0.0.1:53112_    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 67927 INFO  (OverseerCollectionConfigSetProcessor-95353104697655300-127.0.0.1:53112_-n_0000000000) [n:127.0.0.1:53112_    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 67927 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [n:127.0.0.1:53112_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 67928 INFO  (OverseerStateUpdate-95353104697655300-127.0.0.1:53112_-n_0000000000) [n:127.0.0.1:53112_    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 67956 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [n:127.0.0.1:53112_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 67956 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [n:127.0.0.1:53112_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 67957 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [n:127.0.0.1:53112_    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_C2DBDF2397FC8F2D-001/control-001/cores
   [junit4]   2> 67959 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [n:127.0.0.1:53112_    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {loadOnStartup=true, coreNodeName=, configSetProperties=configsetprops.json, transient=false, collection=control_collection, config=solrconfig.xml, shard=, schema=schema.xml, name=collection1, dataDir=data/}
   [junit4]   2> 67959 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [n:127.0.0.1:53112_    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_C2DBDF2397FC8F2D-001/control-001/cores/collection1
   [junit4]   2> 67959 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [n:127.0.0.1:53112_    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 67968 INFO  (coreLoadExecutor-138-thread-1-processing-n:127.0.0.1:53112_) [n:127.0.0.1:53112_ c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 67968 INFO  (coreLoadExecutor-138-thread-1-processing-n:127.0.0.1:53112_) [n:127.0.0.1:53112_ c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 67968 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [n:127.0.0.1:53112_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 67968 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [n:127.0.0.1:53112_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 67972 INFO  (OverseerStateUpdate-95353104697655300-127.0.0.1:53112_-n_0000000000) [n:127.0.0.1:53112_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:53112",
   [junit4]   2>   "node_name":"127.0.0.1:53112_",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "shard":null,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 67972 INFO  (coreLoadExecutor-138-thread-1-processing-n:127.0.0.1:53112_) [n:127.0.0.1:53112_ c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 67973 INFO  (OverseerStateUpdate-95353104697655300-127.0.0.1:53112_-n_0000000000) [n:127.0.0.1:53112_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:53112",
   [junit4]   2>   "node_name":"127.0.0.1:53112_",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "shard":null,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 67973 INFO  (OverseerStateUpdate-95353104697655300-127.0.0.1:53112_-n_0000000000) [n:127.0.0.1:53112_    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 67974 INFO  (OverseerStateUpdate-95353104697655300-127.0.0.1:53112_-n_0000000000) [n:127.0.0.1:53112_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 67977 INFO  (zkCallback-71-thread-1-processing-n:127.0.0.1:53112_) [n:127.0.0.1:53112_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 68973 INFO  (coreLoadExecutor-138-thread-1-processing-n:127.0.0.1:53112_) [n:127.0.0.1:53112_ c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 68973 INFO  (coreLoadExecutor-138-thread-1-processing-n:127.0.0.1:53112_) [n:127.0.0.1:53112_ c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 68974 INFO  (coreLoadExecutor-138-thread-1-processing-n:127.0.0.1:53112_) [n:127.0.0.1:53112_ c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 68974 INFO  (coreLoadExecutor-138-thread-1-processing-n:127.0.0.1:53112_) [n:127.0.0.1:53112_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/control_collection]
   [junit4]   2> 68975 INFO  (coreLoadExecutor-138-thread-1-processing-n:127.0.0.1:53112_) [n:127.0.0.1:53112_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/control_collection] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 68975 INFO  (coreLoadExecutor-138-thread-1-processing-n:127.0.0.1:53112_) [n:127.0.0.1:53112_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_C2DBDF2397FC8F2D-001/control-001/cores/collection1'
   [junit4]   2> 68975 INFO  (coreLoadExecutor-138-thread-1-processing-n:127.0.0.1:53112_) [n:127.0.0.1:53112_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 68975 INFO  (coreLoadExecutor-138-thread-1-processing-n:127.0.0.1:53112_) [n:127.0.0.1:53112_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 68978 INFO  (coreLoadExecutor-138-thread-1-processing-n:127.0.0.1:53112_) [n:127.0.0.1:53112_ c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 68989 INFO  (coreLoadExecutor-138-thread-1-processing-n:127.0.0.1:53112_) [n:127.0.0.1:53112_ c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 69013 INFO  (coreLoadExecutor-138-thread-1-processing-n:127.0.0.1:53112_) [n:127.0.0.1:53112_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.5.0
   [junit4]   2> 69062 INFO  (coreLoadExecutor-138-thread-1-processing-n:127.0.0.1:53112_) [n:127.0.0.1:53112_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 69097 INFO  (coreLoadExecutor-138-thread-1-processing-n:127.0.0.1:53112_) [n:127.0.0.1:53112_ c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 69395 INFO  (coreLoadExecutor-138-thread-1-processing-n:127.0.0.1:53112_) [n:127.0.0.1:53112_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 69407 INFO  (coreLoadExecutor-138-thread-1-processing-n:127.0.0.1:53112_) [n:127.0.0.1:53112_ c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 69409 INFO  (coreLoadExecutor-138-thread-1-processing-n:127.0.0.1:53112_) [n:127.0.0.1:53112_ c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 69434 INFO  (coreLoadExecutor-138-thread-1-processing-n:127.0.0.1:53112_) [n:127.0.0.1:53112_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 69438 INFO  (coreLoadExecutor-138-thread-1-processing-n:127.0.0.1:53112_) [n:127.0.0.1:53112_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 69442 INFO  (coreLoadExecutor-138-thread-1-processing-n:127.0.0.1:53112_) [n:127.0.0.1:53112_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 69443 INFO  (coreLoadExecutor-138-thread-1-processing-n:127.0.0.1:53112_) [n:127.0.0.1:53112_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 69445 INFO  (coreLoadExecutor-138-thread-1-processing-n:127.0.0.1:53112_) [n:127.0.0.1:53112_ c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 69445 INFO  (coreLoadExecutor-138-thread-1-processing-n:127.0.0.1:53112_) [n:127.0.0.1:53112_ c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 69446 INFO  (coreLoadExecutor-138-thread-1-processing-n:127.0.0.1:53112_) [n:127.0.0.1:53112_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.HdfsDirectoryFactory
   [junit4]   2> 69455 INFO  (coreLoadExecutor-138-thread-1-processing-n:127.0.0.1:53112_) [n:127.0.0.1:53112_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:42030/solr_hdfs_home
   [junit4]   2> 69456 INFO  (coreLoadExecutor-138-thread-1-processing-n:127.0.0.1:53112_) [n:127.0.0.1:53112_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
   [junit4]   2> 69456 INFO  (coreLoadExecutor-138-thread-1-processing-n:127.0.0.1:53112_) [n:127.0.0.1:53112_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_C2DBDF2397FC8F2D-001/control-001/cores/collection1], dataDir=[hdfs://localhost:42030/solr_hdfs_home/control_collection/core_node1/data/]
   [junit4]   2> 69456 INFO  (coreLoadExecutor-138-thread-1-processing-n:127.0.0.1:53112_) [n:127.0.0.1:53112_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1347c8ba
   [junit4]   2> 69457 INFO  (coreLoadExecutor-138-thread-1-processing-n:127.0.0.1:53112_) [n:127.0.0.1:53112_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:42030/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 69508 WARN  (coreLoadExecutor-138-thread-1-processing-n:127.0.0.1:53112_) [n:127.0.0.1:53112_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.h.HdfsDirectory The NameNode is in SafeMode - Solr will wait 5 seconds and try again.
   [junit4]   2> 74604 INFO  (coreLoadExecutor-138-thread-1-processing-n:127.0.0.1:53112_) [n:127.0.0.1:53112_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:42030/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 74611 INFO  (coreLoadExecutor-138-thread-1-processing-n:127.0.0.1:53112_) [n:127.0.0.1:53112_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=hdfs://localhost:42030/solr_hdfs_home/control_collection/core_node1/data/index/
   [junit4]   2> 74658 WARN  (coreLoadExecutor-138-thread-1-processing-n:127.0.0.1:53112_) [n:127.0.0.1:53112_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory 'hdfs:/localhost:42030/solr_hdfs_home/control_collection/core_node1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 74658 INFO  (coreLoadExecutor-138-thread-1-processing-n:127.0.0.1:53112_) [n:127.0.0.1:53112_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:42030/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 74675 INFO  (coreLoadExecutor-138-thread-1-processing-n:127.0.0.1:53112_) [n:127.0.0.1:53112_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
   [junit4]   2> 74675 INFO  (coreLoadExecutor-138-thread-1-processing-n:127.0.0.1:53112_) [n:127.0.0.1:53112_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
   [junit4]   2> 74675 INFO  (coreLoadExecutor-138-thread-1-processing-n:127.0.0.1:53112_) [n:127.0.0.1:53112_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
   [junit4]   2> 74738 INFO  (coreLoadExecutor-138-thread-1-processing-n:127.0.0.1:53112_) [n:127.0.0.1:53112_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
   [junit4]   2> 74738 INFO  (coreLoadExecutor-138-thread-1-processing-n:127.0.0.1:53112_) [n:127.0.0.1:53112_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:42030/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 74739 INFO  (coreLoadExecutor-138-thread-1-processing-n:127.0.0.1:53112_) [n:127.0.0.1:53112_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=50, maxMergeAtOnceExplicit=31, maxMergedSegmentMB=2.625, floorSegmentMB=1.6396484375, forceMergeDeletesPctAllowed=1.6719505065398421, segmentsPerTier=25.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.3715057924292058
   [junit4]   2> 75175 INFO  (IPC Server handler 9 on 42030) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:44406 is added to blk_1073741825_1001{blockUCState=COMMITTED, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-aaa45165-7661-4482-9510-794e8d6ca878:NORMAL:127.0.0.1:46526|RBW], ReplicaUnderConstruction[[DISK]DS-ce3f19da-4b31-4e5f-8db9-0034100d9215:NORMAL:127.0.0.1:44406|RBW]]} size 71
   [junit4]   2> 75178 INFO  (IPC Server handler 1 on 42030) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:46526 is added to blk_1073741825_1001 size 71
   [junit4]   2> 75582 INFO  (coreLoadExecutor-138-thread-1-processing-n:127.0.0.1:53112_) [n:127.0.0.1:53112_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(BlockDirectory(HdfsDirectory@hdfs://localhost:42030/solr_hdfs_home/control_collection/core_node1/data/index lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5a861be7); maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 75582 INFO  (coreLoadExecutor-138-thread-1-processing-n:127.0.0.1:53112_) [n:127.0.0.1:53112_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 75585 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:53112_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in hdfs://localhost:42030/solr_hdfs_home/control_collection/core_node1/data/
   [junit4]   2> 75593 INFO  (coreLoadExecutor-138-thread-1-processing-n:127.0.0.1:53112_) [n:127.0.0.1:53112_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 75593 INFO  (coreLoadExecutor-138-thread-1-processing-n:127.0.0.1:53112_) [n:127.0.0.1:53112_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 75593 INFO  (coreLoadExecutor-138-thread-1-processing-n:127.0.0.1:53112_) [n:127.0.0.1:53112_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 75593 INFO  (coreLoadExecutor-138-thread-1-processing-n:127.0.0.1:53112_) [n:127.0.0.1:53112_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 75594 INFO  (coreLoadExecutor-138-thread-1-processing-n:127.0.0.1:53112_) [n:127.0.0.1:53112_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 75594 INFO  (coreLoadExecutor-138-thread-1-processing-n:127.0.0.1:53112_) [n:127.0.0.1:53112_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 75594 INFO  (coreLoadExecutor-138-thread-1-processing-n:127.0.0.1:53112_) [n:127.0.0.1:53112_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 75594 INFO  (coreLoadExecutor-138-thread-1-processing-n:127.0.0.1:53112_) [n:127.0.0.1:53112_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 75594 INFO  (coreLoadExecutor-138-thread-1-processing-n:127.0.0.1:53112_) [n:127.0.0.1:53112_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 75595 WARN  (coreLoadExecutor-138-thread-1-processing-n:127.0.0.1:53112_) [n:127.0.0.1:53112_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
   [junit4]   2> 75598 INFO  (coreLoadExecutor-138-thread-1-processing-n:127.0.0.1:53112_) [n:127.0.0.1:53112_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 75601 INFO  (coreLoadExecutor-138-thread-1-processing-n:127.0.0.1:53112_) [n:127.0.0.1:53112_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 75602 INFO  (coreLoadExecutor-138-thread-1-processing-n:127.0.0.1:53112_) [n:127.0.0.1:53112_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 75604 INFO  (coreLoadExecutor-138-thread-1-processing-n:127.0.0.1:53112_) [n:127.0.0.1:53112_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 75613 INFO  (coreLoadExecutor-138-thread-1-processing-n:127.0.0.1:53112_) [n:127.0.0.1:53112_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /schema,/update/json/docs,/update/csv,/admin/luke,standard,/admin/system,/get,/admin/file,/admin/logging,/replication,/dump,/admin/properties,/update,/config,/admin/plugins,/admin/ping,/admin/segments,/admin/mbeans,/admin/threads,/update/json
   [junit4]   2> 75615 INFO  (coreLoadExecutor-138-thread-1-processing-n:127.0.0.1:53112_) [n:127.0.0.1:53112_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 75618 INFO  (coreLoadExecutor-138-thread-1-processing-n:127.0.0.1:53112_) [n:127.0.0.1:53112_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
   [junit4]   2> 75618 INFO  (coreLoadExecutor-138-thread-1-processing-n:127.0.0.1:53112_) [n:127.0.0.1:53112_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 tlogDfsReplication=2
   [junit4]   2> 75645 INFO  (coreLoadExecutor-138-thread-1-processing-n:127.0.0.1:53112_) [n:127.0.0.1:53112_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 75645 INFO  (coreLoadExecutor-138-thread-1-processing-n:127.0.0.1:53112_) [n:127.0.0.1:53112_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 75651 INFO  (coreLoadExecutor-138-thread-1-processing-n:127.0.0.1:53112_) [n:127.0.0.1:53112_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=2, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.6856081120809738]
   [junit4]   2> 75850 INFO  (coreLoadExecutor-138-thread-1-processing-n:127.0.0.1:53112_) [n:127.0.0.1:53112_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(BlockDirectory(HdfsDirectory@hdfs://localhost:42030/solr_hdfs_home/control_collection/core_node1/data/index lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5a861be7); maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 75850 INFO  (coreLoadExecutor-138-thread-1-processing-n:127.0.0.1:53112_) [n:127.0.0.1:53112_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 75850 INFO  (coreLoadExecutor-138-thread-1-processing-n:127.0.0.1:53112_) [n:127.0.0.1:53112_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@1ff50ed4[collection1] main
   [junit4]   2> 75851 INFO  (coreLoadExecutor-138-thread-1-processing-n:127.0.0.1:53112_) [n:127.0.0.1:53112_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/control_collection]
   [junit4]   2> 75852 INFO  (coreLoadExecutor-138-thread-1-processing-n:127.0.0.1:53112_) [n:127.0.0.1:53112_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/control_collection] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 75852 INFO  (coreLoadExecutor-138-thread-1-processing-n:127.0.0.1:53112_) [n:127.0.0.1:53112_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 75853 INFO  (coreLoadExecutor-138-thread-1-processing-n:127.0.0.1:53112_) [n:127.0.0.1:53112_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 75853 INFO  (coreLoadExecutor-138-thread-1-processing-n:127.0.0.1:53112_) [n:127.0.0.1:53112_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 75853 INFO  (coreLoadExecutor-138-thread-1-processing-n:127.0.0.1:53112_) [n:127.0.0.1:53112_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 75854 INFO  (coreLoadExecutor-138-thread-1-processing-n:127.0.0.1:53112_) [n:127.0.0.1:53112_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 75854 INFO  (coreLoadExecutor-138-thread-1-processing-n:127.0.0.1:53112_) [n:127.0.0.1:53112_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 75855 INFO  (coreLoadExecutor-138-thread-1-processing-n:127.0.0.1:53112_) [n:127.0.0.1:53112_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 75858 INFO  (coreLoadExecutor-138-thread-1-processing-n:127.0.0.1:53112_) [n:127.0.0.1:53112_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 75865 INFO  (coreLoadExecutor-138-thread-1-processing-n:127.0.0.1:53112_) [n:127.0.0.1:53112_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 75868 INFO  (coreLoadExecutor-138-thread-1-processing-n:127.0.0.1:53112_) [n:127.0.0.1:53112_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 256 version buckets from index
   [junit4]   2> 75868 INFO  (coreLoadExecutor-138-thread-1-processing-n:127.0.0.1:53112_) [n:127.0.0.1:53112_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 75868 INFO  (coreLoadExecutor-138-thread-1-processing-n:127.0.0.1:53112_) [n:127.0.0.1:53112_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1525649683892928512
   [junit4]   2> 75868 INFO  (coreLoadExecutor-138-thread-1-processing-n:127.0.0.1:53112_) [n:127.0.0.1:53112_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 2.0ms to seed version buckets with highest version 1525649683892928512
   [junit4]   2> 75869 INFO  (coreLoadExecutor-138-thread-1-processing-n:127.0.0.1:53112_) [n:127.0.0.1:53112_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 75871 INFO  (coreLoadExecutor-138-thread-1-processing-n:127.0.0.1:53112_) [n:127.0.0.1:53112_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 75879 INFO  (searcherExecutor-139-thread-1-processing-c:control_collection n:127.0.0.1:53112_ r:core_node1 x:collection1 s:shard1) [n:127.0.0.1:53112_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@1ff50ed4[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 75884 INFO  (coreZkRegister-132-thread-1-processing-c:control_collection n:127.0.0.1:53112_ r:core_node1 x:collection1 s:shard1) [n:127.0.0.1:53112_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:53112 collection:control_collection shard:shard1
   [junit4]   2> 75892 INFO  (coreZkRegister-132-thread-1-processing-c:control_collection n:127.0.0.1:53112_ r:core_node1 x:collection1 s:shard1) [n:127.0.0.1:53112_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 75903 INFO  (coreZkRegister-132-thread-1-processing-c:control_collection n:127.0.0.1:53112_ r:core_node1 x:collection1 s:shard1) [n:127.0.0.1:53112_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/95353104697655300-core_node1-n_0000000000
   [junit4]   2> 75907 INFO  (coreZkRegister-132-thread-1-processing-c:control_collection n:127.0.0.1:53112_ r:core_node1 x:collection1 s:shard1) [n:127.0.0.1:53112_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard=shard1 and weAreReplacement=false and leaderVoteWait=180000
   [junit4]   2> 75911 INFO  (coreZkRegister-132-thread-1-processing-c:control_collection n:127.0.0.1:53112_ r:core_node1 x:collection1 s:shard1) [n:127.0.0.1:53112_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 75911 INFO  (coreZkRegister-132-thread-1-processing-c:control_collection n:127.0.0.1:53112_ r:core_node1 x:collection1 s:shard1) [n:127.0.0.1:53112_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 75911 INFO  (coreZkRegister-132-thread-1-processing-c:control_collection n:127.0.0.1:53112_ r:core_node1 x:collection1 s:shard1) [n:127.0.0.1:53112_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:53112/collection1/
   [junit4]   2> 75911 INFO  (coreZkRegister-132-thread-1-processing-c:control_collection n:127.0.0.1:53112_ r:core_node1 x:collection1 s:shard1) [n:127.0.0.1:53112_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 75911 INFO  (coreZkRegister-132-thread-1-processing-c:control_collection n:127.0.0.1:53112_ r:core_node1 x:collection1 s:shard1) [n:127.0.0.1:53112_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:53112/collection1/ has no replicas
   [junit4]   2> 75912 INFO  (OverseerStateUpdate-95353104697655300-127.0.0.1:53112_-n_0000000000) [n:127.0.0.1:53112_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 75912 INFO  (coreZkRegister-132-thread-1-processing-c:control_collection n:127.0.0.1:53112_ r:core_node1 x:collection1 s:shard1) [n:127.0.0.1:53112_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 75914 INFO  (zkCallback-71-thread-1-processing-n:127.0.0.1:53112_) [n:127.0.0.1:53112_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 75918 INFO  (coreZkRegister-132-thread-1-processing-c:control_collection n:127.0.0.1:53112_ r:core_node1 x:collection1 s:shard1) [n:127.0.0.1:53112_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 75921 INFO  (coreZkRegister-132-thread-1-processing-c:control_collection n:127.0.0.1:53112_ r:core_node1 x:collection1 s:shard1) [n:127.0.0.1:53112_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:53112/collection1/ shard1
   [junit4]   2> 75922 INFO  (OverseerStateUpdate-95353104697655300-127.0.0.1:53112_-n_0000000000) [n:127.0.0.1:53112_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:53112",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 2
   [junit4]   2> 75972 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 75973 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 75975 INFO  (zkCallback-73-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@47611a87 name:ZooKeeperConnection Watcher:127.0.0.1:43763/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 75975 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 75976 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 75976 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 75983 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 75983 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with stateFormat=2
   [junit4]   2> 75984 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 75984 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 75986 INFO  (zkCallback-74-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@29fb56c name:ZooKeeperConnection Watcher:127.0.0.1:43763/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 75987 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 75987 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 76000 INFO  (OverseerStateUpdate-95353104697655300-127.0.0.1:53112_-n_0000000000) [n:127.0.0.1:53112_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"create",
   [junit4]   2>   "name":"collection1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "stateFormat":"2"} current state version: 2
   [junit4]   2> 76000 INFO  (OverseerStateUpdate-95353104697655300-127.0.0.1:53112_-n_0000000000) [n:127.0.0.1:53112_    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 76004 INFO  (zkCallback-73-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 76004 INFO  (zkCallback-71-thread-1-processing-n:127.0.0.1:53112_) [n:127.0.0.1:53112_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 76022 INFO  (coreZkRegister-132-thread-1-processing-c:control_collection n:127.0.0.1:53112_ r:core_node1 x:collection1 s:shard1) [n:127.0.0.1:53112_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:53112/collection1/ and leader is http://127.0.0.1:53112/collection1/
   [junit4]   2> 76022 INFO  (coreZkRegister-132-thread-1-processing-c:control_collection n:127.0.0.1:53112_ r:core_node1 x:collection1 s:shard1) [n:127.0.0.1:53112_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:53112
   [junit4]   2> 76022 INFO  (coreZkRegister-132-thread-1-processing-c:control_collection n:127.0.0.1:53112_ r:core_node1 x:collection1 s:shard1) [n:127.0.0.1:53112_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 76023 INFO  (coreZkRegister-132-thread-1-processing-c:control_collection n:127.0.0.1:53112_ r:core_node1 x:collection1 s:shard1) [n:127.0.0.1:53112_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 76023 INFO  (coreZkRegister-132-thread-1-processing-c:control_collection n:127.0.0.1:53112_ r:core_node1 x:collection1 s:shard1) [n:127.0.0.1:53112_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 76030 INFO  (OverseerStateUpdate-95353104697655300-127.0.0.1:53112_-n_0000000000) [n:127.0.0.1:53112_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:53112",
   [junit4]   2>   "node_name":"127.0.0.1:53112_",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "operation":"state"} current state version: 3
   [junit4]   2> 76031 INFO  (OverseerStateUpdate-95353104697655300-127.0.0.1:53112_-n_0000000000) [n:127.0.0.1:53112_    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:53112",
   [junit4]   2>   "node_name":"127.0.0.1:53112_",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 76032 INFO  (OverseerStateUpdate-95353104697655300-127.0.0.1:53112_-n_0000000000) [n:127.0.0.1:53112_    ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/collection1/state.json
   [junit4]   2> 76131 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_C2DBDF2397FC8F2D-001/shard-1-001/cores/collection1
   [junit4]   2> 76134 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_C2DBDF2397FC8F2D-001/shard-1-001
   [junit4]   2> 76135 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 76139 INFO  (zkCallback-71-thread-2-processing-n:127.0.0.1:53112_) [n:127.0.0.1:53112_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 76140 INFO  (zkCallback-73-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 76146 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@22184183{/,null,AVAILABLE}
   [junit4]   2> 76147 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [    ] o.e.j.s.ServerConnector Started ServerConnector@667510f1{HTTP/1.1}{127.0.0.1:46053}
   [junit4]   2> 76147 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [    ] o.e.j.s.Server Started @79278ms
   [junit4]   2> 76147 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solrconfig=solrconfig.xml, hostPort=46053, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_C2DBDF2397FC8F2D-001/shard-1-001/cores, hostContext=/}
   [junit4]   2> 76148 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@7b3cb2c6
   [junit4]   2> 76148 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_C2DBDF2397FC8F2D-001/shard-1-001'
   [junit4]   2> 76148 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 76149 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 76149 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 76163 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 76164 INFO  (zkCallback-75-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7bc3ac31 name:ZooKeeperConnection Watcher:127.0.0.1:43763/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 76164 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 76164 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 76170 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 76170 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_C2DBDF2397FC8F2D-001/shard-1-001/solr.xml
   [junit4]   2> 76196 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_C2DBDF2397FC8F2D-001/shard-1-001/cores
   [junit4]   2> 76196 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [    ] o.a.s.c.CoreContainer New CoreContainer 1580127913
   [junit4]   2> 76196 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_C2DBDF2397FC8F2D-001/shard-1-001]
   [junit4]   2> 76196 WARN  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_C2DBDF2397FC8F2D-001/shard-1-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_C2DBDF2397FC8F2D-001/shard-1-001/lib
   [junit4]   2> 76197 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 76199 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 76200 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 76200 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 76200 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:43763/solr
   [junit4]   2> 76200 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 76200 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 76201 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 76202 INFO  (zkCallback-78-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@495ee3a2 name:ZooKeeperConnection Watcher:127.0.0.1:43763 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 76203 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 76203 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 76215 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [n:127.0.0.1:46053_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 76216 INFO  (zkCallback-79-thread-1-processing-n:127.0.0.1:46053_) [n:127.0.0.1:46053_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@44d47e7f name:ZooKeeperConnection Watcher:127.0.0.1:43763/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 76216 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [n:127.0.0.1:46053_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 76223 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [n:127.0.0.1:46053_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 77236 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [n:127.0.0.1:46053_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:46053_
   [junit4]   2> 77237 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [n:127.0.0.1:46053_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:46053_
   [junit4]   2> 77238 INFO  (zkCallback-71-thread-2-processing-n:127.0.0.1:53112_) [n:127.0.0.1:53112_    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 77239 INFO  (zkCallback-73-thread-1) [    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 77240 INFO  (zkCallback-79-thread-1-processing-n:127.0.0.1:46053_) [n:127.0.0.1:46053_    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 77242 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [n:127.0.0.1:46053_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 77243 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [n:127.0.0.1:46053_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/95353104697655305-127.0.0.1:46053_-n_0000000001
   [junit4]   2> 77244 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [n:127.0.0.1:46053_    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/95353104697655300-127.0.0.1:53112_-n_0000000000 to know if I could be the leader
   [junit4]   2> 77268 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [n:127.0.0.1:46053_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 77268 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [n:127.0.0.1:46053_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 77269 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [n:127.0.0.1:46053_    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_C2DBDF2397FC8F2D-001/shard-1-001/cores
   [junit4]   2> 77270 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [n:127.0.0.1:46053_    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {configSetProperties=configsetprops.json, collection=collection1, dataDir=data/, shard=, loadOnStartup=true, transient=false, schema=schema.xml, name=collection1, coreNodeName=, config=solrconfig.xml}
   [junit4]   2> 77270 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [n:127.0.0.1:46053_    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_C2DBDF2397FC8F2D-001/shard-1-001/cores/collection1
   [junit4]   2> 77270 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [n:127.0.0.1:46053_    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 77339 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [n:127.0.0.1:46053_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 77339 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[C2DBDF2397FC8F2D]) [n:127.0.0.1:46053_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 77339 INFO  (coreLoadExecutor-149-thread-1-processing-n:127.0.0.1:46053_) [n:127.0.0.1:46053_ c:collection1   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 77340 INFO  (coreLoadExecutor-149-thread-1-processing-n:127.0.0.1:46053_) [n:127.0.0.1:46053_ c:collection1   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 77342 INFO  (coreLoadExecutor-149-thread-1-processing-n:127.0.0.1:46053_) [n:127.0.0.1:46053_ c:collection1   x:collection1] o.a.s.c.ZkController Registering watch for collection collection1
   [junit4]   2> 77342 INFO  (coreLoadExecutor-149-thread-1-processing-n:127.0.0.1:46053_) [n:127.0.0.1:46053_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader addZkWatch [collection1]
   [junit4]   2> 77343 INFO  (OverseerStateUpdate-95353104697655300-127.0.0.1:53112_-n_0000000000) [n:127.0.0.1:53112_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "base_url":"http://127.0.0.1:46053",
   [junit4]   2>   "node_name":"127.0.0.1:46053_",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "shard":null,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "operation":"state"} current state version: 4
   [junit4]   2> 77343 INFO  (coreLoadExecutor-149-thread-1-processing-n:127.0.0.1:46053_) [n:127.0.0.1:46053_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Add data for [collection1] ver [0]
   [junit4]   2> 77343 INFO  (coreLoadExecutor-149-thread-1-processing-n:127.0.0.1:46053_) [n:127.0.0.1:46053_ c:collection1   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 77343 INFO  (OverseerStateUpdate-95353104697655300-127.0.0.1:53112_-n_0000000000) [n:127.0.0.1:53112_    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "base_url":"http://127.0.0.1:46053",
   [junit4]   2>   "node_name":"127.0.0.1:46053_",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "shard":null,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 77343 INFO  (OverseerStateUpdate-95353104697655300-127.0.0.1:53112_-n_0000000000) [n:127.0.0.1:53112_    ] o.a.s.c.o.ReplicaMutator Collection already exists with numShards=2
   [junit4]   2> 77344 INFO  (OverseerStateUpdate-95353104697655300-127.0.0.1:53112_-n_0000000000) [n:127.0.0.1:53112_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2
   [junit4]   2> 77457 INFO  (OverseerStateUpdate-95353104697655300-127.0.0.1:53112_-n_0000000000) [n:127.0.0.1:53112_    ] o.a.s.c.o.ZkStateWriter going to update_collection /collections/collection1/state.json version: 0
   [junit4]   2> 77458 INFO  (zkCallback-79-thread-1-processing-n:127.0.0.1:46053_) [n:127.0.0.1:46053_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 77459 INFO  (zkCallback-79-thread-1-processing-n:127.0.0.1:46053_) [n:127.0.0.1:46053_    ] o.a.s.c.c.ZkStateReader Updating data for [collection1] from [0] to [1]
   [junit4]   2> 78344 INFO  (coreLoadExecutor-149-thread-1-processing-n:127.0.0.1:46053_) [n:127.0.0.1:46053_ c:collection1   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 78344 INFO  (coreLoadExecutor-149-thread-1-processing-n:127.0.0.1:46053_) [n:127.0.0.1:46053_ c:collection1   x:collection1] o.a.s.c.ZkController Check for collection zkNode:collection1
   [junit4]   2> 78345 INFO  (coreLoadExecutor-149-thread-1-processing-n:127.0.0.1:46053_) [n:127.0.0.1:46053_ c:collection1   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 78345 INFO  (coreLoadExecutor-149-thread-1-processing-n:127.0.0.1:46053_) [n:127.0.0.1:46053_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/collection1]
   [junit4]   2> 78346 INFO  (coreLoadExecutor-149-thread-1-processing-n:127.0.0.1:46053_) [n:127.0.0.1:46053_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/collection1] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 78346 INFO  (coreLoadExecutor-149-thread-1-processing-n:127.0.0.1:46053_) [n:127.0.0.1:46053_ c:collection1   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_C2DBDF2397FC8F2D-001/shard-1-001/cores/collection1'
   [junit4]   2> 78346 INFO  (coreLoadExecutor-149-thread-1-processing-n:127.0.0.1:46053_) [n:127.0.0.1:46053_ c:collection1   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 78346 INFO  (coreLoadExecutor-149-thread-1-processing-n:127.0.0.1:46053_) [n:127.0.0.1:46053_ c:collection1   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system

[...truncated too long message...]

host/127.0.0.1:42030
   [junit4]   2> 298528 WARN  (org.apache.hadoop.hdfs.server.blockmanagement.DecommissionManager$Monitor@621f1599) [    ] o.a.h.h.s.b.DecommissionManager Monitor interrupted: java.lang.InterruptedException: sleep interrupted
   [junit4]   2> 298560 INFO  (SUITE-SharedFSAutoReplicaFailoverTest-seed#[C2DBDF2397FC8F2D]-worker) [n:127.0.0.1:39931_ c:solrj_collection3 s:shard1 r:core_node2 x:solrj_collection3_shard1_replica1] o.m.log Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0
   [junit4]   2> 298664 ERROR (SUITE-SharedFSAutoReplicaFailoverTest-seed#[C2DBDF2397FC8F2D]-worker) [n:127.0.0.1:39931_ c:solrj_collection3 s:shard1 r:core_node2 x:solrj_collection3_shard1_replica1] o.a.h.m.l.MethodMetric Error invoking method getBlocksTotal
   [junit4]   2> java.lang.reflect.InvocationTargetException
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   [junit4]   2> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2> 	at java.lang.reflect.Method.invoke(Method.java:606)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.MethodMetric$2.snapshot(MethodMetric.java:111)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.MethodMetric.snapshot(MethodMetric.java:144)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.MetricsRegistry.snapshot(MetricsRegistry.java:387)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.MetricsSourceBuilder$1.getMetrics(MetricsSourceBuilder.java:79)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.getMetrics(MetricsSourceAdapter.java:195)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.updateJmxCache(MetricsSourceAdapter.java:172)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.getMBeanInfo(MetricsSourceAdapter.java:151)
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getClassName(DefaultMBeanServerInterceptor.java:1804)
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.safeGetClassName(DefaultMBeanServerInterceptor.java:1595)
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.checkMBeanPermission(DefaultMBeanServerInterceptor.java:1813)
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.exclusiveUnregisterMBean(DefaultMBeanServerInterceptor.java:430)
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.unregisterMBean(DefaultMBeanServerInterceptor.java:415)
   [junit4]   2> 	at com.sun.jmx.mbeanserver.JmxMBeanServer.unregisterMBean(JmxMBeanServer.java:546)
   [junit4]   2> 	at org.apache.hadoop.metrics2.util.MBeans.unregister(MBeans.java:81)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.stopMBeans(MetricsSourceAdapter.java:227)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.stop(MetricsSourceAdapter.java:212)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.stopSources(MetricsSystemImpl.java:461)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.stop(MetricsSystemImpl.java:212)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.shutdown(MetricsSystemImpl.java:592)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.shutdownInstance(DefaultMetricsSystem.java:72)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.shutdown(DefaultMetricsSystem.java:68)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.metrics.NameNodeMetrics.shutdown(NameNodeMetrics.java:145)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.NameNode.stop(NameNode.java:822)
   [junit4]   2> 	at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1720)
   [junit4]   2> 	at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1699)
   [junit4]   2> 	at org.apache.solr.cloud.hdfs.HdfsTestUtil.teardownClass(HdfsTestUtil.java:198)
   [junit4]   2> 	at org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.hdfsFailoverAfterClass(SharedFSAutoReplicaFailoverTest.java:82)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   [junit4]   2> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2> 	at java.lang.reflect.Method.invoke(Method.java:606)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:834)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> Caused by: java.lang.NullPointerException
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.blockmanagement.BlocksMap.size(BlocksMap.java:198)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.getTotalBlocks(BlockManager.java:3291)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlocksTotal(FSNamesystem.java:6223)
   [junit4]   2> 	... 54 more
   [junit4]   2> 298686 INFO  (SUITE-SharedFSAutoReplicaFailoverTest-seed#[C2DBDF2397FC8F2D]-worker) [n:127.0.0.1:39931_ c:solrj_collection3 s:shard1 r:core_node2 x:solrj_collection3_shard1_replica1] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_C2DBDF2397FC8F2D-001
   [junit4]   2> Feb 08, 2016 11:12:22 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 2 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=HighCompressionCompressingStoredFields(storedFieldsFormat=CompressingStoredFieldsFormat(compressionMode=HIGH_COMPRESSION, chunkSize=4, maxDocsPerChunk=162, blockSize=480), termVectorsFormat=CompressingTermVectorsFormat(compressionMode=HIGH_COMPRESSION, chunkSize=4, blockSize=480)), sim=DefaultSimilarity, locale=ar-IQ, timezone=America/Guadeloupe
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_80 (64-bit)/cpus=4,threads=3,free=66900608,total=348127232
   [junit4]   2> NOTE: All tests run in this JVM: [PKIAuthenticationIntegrationTest, TestShortCircuitedRequests, AnalysisErrorHandlingTest, SharedFSAutoReplicaFailoverTest]
   [junit4] Completed [59/566 (1!)] on J1 in 247.23s, 1 test, 1 failure <<< FAILURES!

[...truncated 1662 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:757: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:694: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:534: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1477: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1033: There were test failures: 566 suites (7 ignored), 2268 tests, 1 failure, 33 ignored (18 assumptions) [seed: C2DBDF2397FC8F2D]

Total time: 207 minutes 28 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1091 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1091/

1 tests failed.
FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=15770, name=collection0, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=15770, name=collection0, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: java.lang.RuntimeException: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:51622, http://127.0.0.1:56566, http://127.0.0.1:52691, http://127.0.0.1:60037, http://127.0.0.1:34097]
	at __randomizedtesting.SeedInfo.seed([7AF4185537C08820]:0)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:890)
Caused by: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:51622, http://127.0.0.1:56566, http://127.0.0.1:52691, http://127.0.0.1:60037, http://127.0.0.1:34097]
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:352)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1573)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1594)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:880)
Caused by: org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:56566
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:585)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:240)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:229)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	... 7 more
Caused by: java.net.SocketTimeoutException: Read timed out
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.read(SocketInputStream.java:152)
	at java.net.SocketInputStream.read(SocketInputStream.java:122)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:160)
	at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:84)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:273)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
	at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
	at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)
	at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)
	at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197)
	at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)
	at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)
	at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685)
	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487)
	at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:481)
	... 11 more




Build Log:
[...truncated 11997 lines...]
   [junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_7AF4185537C08820-001/init-core-data-001
   [junit4]   2> 1588672 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[7AF4185537C08820]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false)
   [junit4]   2> 1588672 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[7AF4185537C08820]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   2> 1588676 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 1588677 INFO  (Thread-9114) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1588677 INFO  (Thread-9114) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 1588777 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [    ] o.a.s.c.ZkTestServer start zk server on port:48757
   [junit4]   2> 1588777 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1588778 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1588786 INFO  (zkCallback-1392-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1ad96fd1 name:ZooKeeperConnection Watcher:127.0.0.1:48757 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1588786 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1588786 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1588786 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 1588788 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1588790 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1588792 INFO  (zkCallback-1393-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@27b62296 name:ZooKeeperConnection Watcher:127.0.0.1:48757/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1588792 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1588792 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1588792 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 1588795 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 1588797 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 1588798 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 1588799 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 1588800 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 1588802 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 1588802 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 1588804 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1588804 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1588806 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 1588806 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 1588807 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 1588807 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 1588809 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 1588809 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 1588814 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 1588814 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 1588815 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1588816 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1588817 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1588817 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1588819 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 1588819 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 1588821 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 1588821 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 1588919 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_7AF4185537C08820-001/control-001/cores/collection1
   [junit4]   2> 1588920 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 1588922 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@50cfb1d0{/,null,AVAILABLE}
   [junit4]   2> 1588930 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [    ] o.e.j.s.ServerConnector Started ServerConnector@73a2f822{HTTP/1.1}{127.0.0.1:60037}
   [junit4]   2> 1588930 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [    ] o.e.j.s.Server Started @1591596ms
   [junit4]   2> 1588930 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_7AF4185537C08820-001/control-001/cores, hostContext=/, hostPort=60037, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_7AF4185537C08820-001/tempDir-001/control/data}
   [junit4]   2> 1588931 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@4f6de641
   [junit4]   2> 1588931 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_7AF4185537C08820-001/control-001'
   [junit4]   2> 1588931 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1588931 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1588931 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1588933 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1588933 INFO  (zkCallback-1394-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@24d1d7b4 name:ZooKeeperConnection Watcher:127.0.0.1:48757/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1588933 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1588934 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1588935 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1588935 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_7AF4185537C08820-001/control-001/solr.xml
   [junit4]   2> 1588950 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_7AF4185537C08820-001/control-001/cores
   [junit4]   2> 1588950 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [    ] o.a.s.c.CoreContainer New CoreContainer 1586151072
   [junit4]   2> 1588950 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_7AF4185537C08820-001/control-001]
   [junit4]   2> 1588950 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_7AF4185537C08820-001/control-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_7AF4185537C08820-001/control-001/lib
   [junit4]   2> 1588950 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1588952 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1588952 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1588952 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1588952 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:48757/solr
   [junit4]   2> 1588953 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1588953 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1588954 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1588954 INFO  (zkCallback-1397-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3fa37131 name:ZooKeeperConnection Watcher:127.0.0.1:48757 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1588955 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1588955 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1588958 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [n:127.0.0.1:60037_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1588959 INFO  (zkCallback-1398-thread-1-processing-n:127.0.0.1:60037_) [n:127.0.0.1:60037_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6905f881 name:ZooKeeperConnection Watcher:127.0.0.1:48757/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1588959 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [n:127.0.0.1:60037_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1588960 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [n:127.0.0.1:60037_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 1588962 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [n:127.0.0.1:60037_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 1588965 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [n:127.0.0.1:60037_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 1588967 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [n:127.0.0.1:60037_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 1588969 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [n:127.0.0.1:60037_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 1588971 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [n:127.0.0.1:60037_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 1588973 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [n:127.0.0.1:60037_    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 1588974 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [n:127.0.0.1:60037_    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 1588975 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [n:127.0.0.1:60037_    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 1588976 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [n:127.0.0.1:60037_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:60037_
   [junit4]   2> 1588976 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [n:127.0.0.1:60037_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:60037_
   [junit4]   2> 1588978 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [n:127.0.0.1:60037_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 1588978 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [n:127.0.0.1:60037_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 1588980 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [n:127.0.0.1:60037_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1588980 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [n:127.0.0.1:60037_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/95347554668707844-127.0.0.1:60037_-n_0000000000
   [junit4]   2> 1588980 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [n:127.0.0.1:60037_    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:60037_
   [junit4]   2> 1588980 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [n:127.0.0.1:60037_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 1588981 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [n:127.0.0.1:60037_    ] o.a.s.c.Overseer Overseer (id=95347554668707844-127.0.0.1:60037_-n_0000000000) starting
   [junit4]   2> 1588983 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [n:127.0.0.1:60037_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 1588988 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [n:127.0.0.1:60037_    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 1588994 INFO  (OverseerCollectionConfigSetProcessor-95347554668707844-127.0.0.1:60037_-n_0000000000) [n:127.0.0.1:60037_    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 1588994 INFO  (OverseerStateUpdate-95347554668707844-127.0.0.1:60037_-n_0000000000) [n:127.0.0.1:60037_    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 1588994 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [n:127.0.0.1:60037_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1589015 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [n:127.0.0.1:60037_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1589015 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [n:127.0.0.1:60037_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1589016 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [n:127.0.0.1:60037_    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_7AF4185537C08820-001/control-001/cores
   [junit4]   2> 1589018 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [n:127.0.0.1:60037_    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {collection=control_collection, configSetProperties=configsetprops.json, loadOnStartup=true, schema=schema.xml, dataDir=data/, transient=false, coreNodeName=, shard=, config=solrconfig.xml, name=collection1}
   [junit4]   2> 1589018 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [n:127.0.0.1:60037_    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_7AF4185537C08820-001/control-001/cores/collection1
   [junit4]   2> 1589018 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [n:127.0.0.1:60037_    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 1589025 INFO  (coreLoadExecutor-3660-thread-1-processing-n:127.0.0.1:60037_) [n:127.0.0.1:60037_ c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 1589025 INFO  (coreLoadExecutor-3660-thread-1-processing-n:127.0.0.1:60037_) [n:127.0.0.1:60037_ c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1589025 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [n:127.0.0.1:60037_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 1589025 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [n:127.0.0.1:60037_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1589027 INFO  (coreLoadExecutor-3660-thread-1-processing-n:127.0.0.1:60037_) [n:127.0.0.1:60037_ c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 1589027 INFO  (OverseerStateUpdate-95347554668707844-127.0.0.1:60037_-n_0000000000) [n:127.0.0.1:60037_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "node_name":"127.0.0.1:60037_",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:60037",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "roles":null} current state version: 0
   [junit4]   2> 1589028 INFO  (OverseerStateUpdate-95347554668707844-127.0.0.1:60037_-n_0000000000) [n:127.0.0.1:60037_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "node_name":"127.0.0.1:60037_",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:60037",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "roles":null}
   [junit4]   2> 1589028 INFO  (OverseerStateUpdate-95347554668707844-127.0.0.1:60037_-n_0000000000) [n:127.0.0.1:60037_    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 1589028 INFO  (OverseerStateUpdate-95347554668707844-127.0.0.1:60037_-n_0000000000) [n:127.0.0.1:60037_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 1589030 INFO  (zkCallback-1398-thread-1-processing-n:127.0.0.1:60037_) [n:127.0.0.1:60037_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1590027 INFO  (coreLoadExecutor-3660-thread-1-processing-n:127.0.0.1:60037_) [n:127.0.0.1:60037_ c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1590027 INFO  (coreLoadExecutor-3660-thread-1-processing-n:127.0.0.1:60037_) [n:127.0.0.1:60037_ c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 1590028 INFO  (coreLoadExecutor-3660-thread-1-processing-n:127.0.0.1:60037_) [n:127.0.0.1:60037_ c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 1590028 INFO  (coreLoadExecutor-3660-thread-1-processing-n:127.0.0.1:60037_) [n:127.0.0.1:60037_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/control_collection]
   [junit4]   2> 1590028 INFO  (coreLoadExecutor-3660-thread-1-processing-n:127.0.0.1:60037_) [n:127.0.0.1:60037_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/control_collection] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 1590028 INFO  (coreLoadExecutor-3660-thread-1-processing-n:127.0.0.1:60037_) [n:127.0.0.1:60037_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_7AF4185537C08820-001/control-001/cores/collection1'
   [junit4]   2> 1590029 INFO  (coreLoadExecutor-3660-thread-1-processing-n:127.0.0.1:60037_) [n:127.0.0.1:60037_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1590029 INFO  (coreLoadExecutor-3660-thread-1-processing-n:127.0.0.1:60037_) [n:127.0.0.1:60037_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1590031 INFO  (coreLoadExecutor-3660-thread-1-processing-n:127.0.0.1:60037_) [n:127.0.0.1:60037_ c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 1590037 INFO  (coreLoadExecutor-3660-thread-1-processing-n:127.0.0.1:60037_) [n:127.0.0.1:60037_ c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 1590052 INFO  (coreLoadExecutor-3660-thread-1-processing-n:127.0.0.1:60037_) [n:127.0.0.1:60037_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.5.0
   [junit4]   2> 1590086 INFO  (coreLoadExecutor-3660-thread-1-processing-n:127.0.0.1:60037_) [n:127.0.0.1:60037_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1590094 INFO  (coreLoadExecutor-3660-thread-1-processing-n:127.0.0.1:60037_) [n:127.0.0.1:60037_ c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1590385 INFO  (coreLoadExecutor-3660-thread-1-processing-n:127.0.0.1:60037_) [n:127.0.0.1:60037_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 1590394 INFO  (coreLoadExecutor-3660-thread-1-processing-n:127.0.0.1:60037_) [n:127.0.0.1:60037_ c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 1590396 INFO  (coreLoadExecutor-3660-thread-1-processing-n:127.0.0.1:60037_) [n:127.0.0.1:60037_ c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 1590408 INFO  (coreLoadExecutor-3660-thread-1-processing-n:127.0.0.1:60037_) [n:127.0.0.1:60037_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1590411 INFO  (coreLoadExecutor-3660-thread-1-processing-n:127.0.0.1:60037_) [n:127.0.0.1:60037_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1590414 INFO  (coreLoadExecutor-3660-thread-1-processing-n:127.0.0.1:60037_) [n:127.0.0.1:60037_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1590414 INFO  (coreLoadExecutor-3660-thread-1-processing-n:127.0.0.1:60037_) [n:127.0.0.1:60037_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1590416 INFO  (coreLoadExecutor-3660-thread-1-processing-n:127.0.0.1:60037_) [n:127.0.0.1:60037_ c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 1590416 INFO  (coreLoadExecutor-3660-thread-1-processing-n:127.0.0.1:60037_) [n:127.0.0.1:60037_ c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 1590416 INFO  (coreLoadExecutor-3660-thread-1-processing-n:127.0.0.1:60037_) [n:127.0.0.1:60037_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 1590417 INFO  (coreLoadExecutor-3660-thread-1-processing-n:127.0.0.1:60037_) [n:127.0.0.1:60037_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_7AF4185537C08820-001/control-001/cores/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_7AF4185537C08820-001/control-001/cores/collection1/data/]
   [junit4]   2> 1590417 INFO  (coreLoadExecutor-3660-thread-1-processing-n:127.0.0.1:60037_) [n:127.0.0.1:60037_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@2ad628c1
   [junit4]   2> 1590417 INFO  (coreLoadExecutor-3660-thread-1-processing-n:127.0.0.1:60037_) [n:127.0.0.1:60037_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_7AF4185537C08820-001/control-001/cores/collection1/data
   [junit4]   2> 1590418 INFO  (coreLoadExecutor-3660-thread-1-processing-n:127.0.0.1:60037_) [n:127.0.0.1:60037_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_7AF4185537C08820-001/control-001/cores/collection1/data/index/
   [junit4]   2> 1590418 WARN  (coreLoadExecutor-3660-thread-1-processing-n:127.0.0.1:60037_) [n:127.0.0.1:60037_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_7AF4185537C08820-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 1590418 INFO  (coreLoadExecutor-3660-thread-1-processing-n:127.0.0.1:60037_) [n:127.0.0.1:60037_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_7AF4185537C08820-001/control-001/cores/collection1/data/index
   [junit4]   2> 1590419 INFO  (coreLoadExecutor-3660-thread-1-processing-n:127.0.0.1:60037_) [n:127.0.0.1:60037_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=1467220895, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
   [junit4]   2> 1590419 INFO  (coreLoadExecutor-3660-thread-1-processing-n:127.0.0.1:60037_) [n:127.0.0.1:60037_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@36dc4890 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@38200213),segFN=segments_1,generation=1}
   [junit4]   2> 1590419 INFO  (coreLoadExecutor-3660-thread-1-processing-n:127.0.0.1:60037_) [n:127.0.0.1:60037_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 1590419 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:60037_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_7AF4185537C08820-001/control-001/cores/collection1/data/
   [junit4]   2> 1590420 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:60037_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_7AF4185537C08820-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 1590423 INFO  (coreLoadExecutor-3660-thread-1-processing-n:127.0.0.1:60037_) [n:127.0.0.1:60037_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 1590423 INFO  (coreLoadExecutor-3660-thread-1-processing-n:127.0.0.1:60037_) [n:127.0.0.1:60037_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 1590423 INFO  (coreLoadExecutor-3660-thread-1-processing-n:127.0.0.1:60037_) [n:127.0.0.1:60037_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 1590424 INFO  (coreLoadExecutor-3660-thread-1-processing-n:127.0.0.1:60037_) [n:127.0.0.1:60037_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1590424 INFO  (coreLoadExecutor-3660-thread-1-processing-n:127.0.0.1:60037_) [n:127.0.0.1:60037_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1590424 INFO  (coreLoadExecutor-3660-thread-1-processing-n:127.0.0.1:60037_) [n:127.0.0.1:60037_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 1590424 INFO  (coreLoadExecutor-3660-thread-1-processing-n:127.0.0.1:60037_) [n:127.0.0.1:60037_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1590424 INFO  (coreLoadExecutor-3660-thread-1-processing-n:127.0.0.1:60037_) [n:127.0.0.1:60037_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1590424 INFO  (coreLoadExecutor-3660-thread-1-processing-n:127.0.0.1:60037_) [n:127.0.0.1:60037_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 1590424 WARN  (coreLoadExecutor-3660-thread-1-processing-n:127.0.0.1:60037_) [n:127.0.0.1:60037_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
   [junit4]   2> 1590427 INFO  (coreLoadExecutor-3660-thread-1-processing-n:127.0.0.1:60037_) [n:127.0.0.1:60037_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1590428 INFO  (coreLoadExecutor-3660-thread-1-processing-n:127.0.0.1:60037_) [n:127.0.0.1:60037_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1590429 INFO  (coreLoadExecutor-3660-thread-1-processing-n:127.0.0.1:60037_) [n:127.0.0.1:60037_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1590431 INFO  (coreLoadExecutor-3660-thread-1-processing-n:127.0.0.1:60037_) [n:127.0.0.1:60037_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1590434 INFO  (coreLoadExecutor-3660-thread-1-processing-n:127.0.0.1:60037_) [n:127.0.0.1:60037_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/luke,/update/json/docs,/config,/update/json,/get,/update,/dump,/update/csv,/admin/system,/replication,/admin/mbeans,/admin/segments,/admin/threads,/admin/logging,/admin/file,standard,/admin/ping,/admin/plugins,/schema,/admin/properties
   [junit4]   2> 1590436 INFO  (coreLoadExecutor-3660-thread-1-processing-n:127.0.0.1:60037_) [n:127.0.0.1:60037_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 1590436 INFO  (coreLoadExecutor-3660-thread-1-processing-n:127.0.0.1:60037_) [n:127.0.0.1:60037_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1590436 INFO  (coreLoadExecutor-3660-thread-1-processing-n:127.0.0.1:60037_) [n:127.0.0.1:60037_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1590437 INFO  (coreLoadExecutor-3660-thread-1-processing-n:127.0.0.1:60037_) [n:127.0.0.1:60037_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1590437 INFO  (coreLoadExecutor-3660-thread-1-processing-n:127.0.0.1:60037_) [n:127.0.0.1:60037_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1590438 INFO  (coreLoadExecutor-3660-thread-1-processing-n:127.0.0.1:60037_) [n:127.0.0.1:60037_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=50, maxMergeAtOnceExplicit=30, maxMergedSegmentMB=49.1640625, floorSegmentMB=0.25, forceMergeDeletesPctAllowed=25.173372786555813, segmentsPerTier=46.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.7448951959595609
   [junit4]   2> 1590438 INFO  (coreLoadExecutor-3660-thread-1-processing-n:127.0.0.1:60037_) [n:127.0.0.1:60037_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@36dc4890 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@38200213),segFN=segments_1,generation=1}
   [junit4]   2> 1590438 INFO  (coreLoadExecutor-3660-thread-1-processing-n:127.0.0.1:60037_) [n:127.0.0.1:60037_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 1590438 INFO  (coreLoadExecutor-3660-thread-1-processing-n:127.0.0.1:60037_) [n:127.0.0.1:60037_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@4d627d06[collection1] main
   [junit4]   2> 1590438 INFO  (coreLoadExecutor-3660-thread-1-processing-n:127.0.0.1:60037_) [n:127.0.0.1:60037_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/control_collection]
   [junit4]   2> 1590439 INFO  (coreLoadExecutor-3660-thread-1-processing-n:127.0.0.1:60037_) [n:127.0.0.1:60037_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/control_collection] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 1590439 INFO  (coreLoadExecutor-3660-thread-1-processing-n:127.0.0.1:60037_) [n:127.0.0.1:60037_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 1590439 INFO  (coreLoadExecutor-3660-thread-1-processing-n:127.0.0.1:60037_) [n:127.0.0.1:60037_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1590440 INFO  (coreLoadExecutor-3660-thread-1-processing-n:127.0.0.1:60037_) [n:127.0.0.1:60037_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 1590440 INFO  (coreLoadExecutor-3660-thread-1-processing-n:127.0.0.1:60037_) [n:127.0.0.1:60037_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1590440 INFO  (coreLoadExecutor-3660-thread-1-processing-n:127.0.0.1:60037_) [n:127.0.0.1:60037_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 1590440 INFO  (coreLoadExecutor-3660-thread-1-processing-n:127.0.0.1:60037_) [n:127.0.0.1:60037_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1590440 INFO  (coreLoadExecutor-3660-thread-1-processing-n:127.0.0.1:60037_) [n:127.0.0.1:60037_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 1590440 INFO  (coreLoadExecutor-3660-thread-1-processing-n:127.0.0.1:60037_) [n:127.0.0.1:60037_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1590441 INFO  (searcherExecutor-3661-thread-1-processing-c:control_collection n:127.0.0.1:60037_ x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:60037_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@4d627d06[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1590441 INFO  (coreLoadExecutor-3660-thread-1-processing-n:127.0.0.1:60037_) [n:127.0.0.1:60037_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 1590442 INFO  (coreLoadExecutor-3660-thread-1-processing-n:127.0.0.1:60037_) [n:127.0.0.1:60037_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 1590442 INFO  (coreLoadExecutor-3660-thread-1-processing-n:127.0.0.1:60037_) [n:127.0.0.1:60037_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 1590442 INFO  (coreLoadExecutor-3660-thread-1-processing-n:127.0.0.1:60037_) [n:127.0.0.1:60037_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1525560876549013504
   [junit4]   2> 1590444 INFO  (coreLoadExecutor-3660-thread-1-processing-n:127.0.0.1:60037_) [n:127.0.0.1:60037_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 2.0ms to seed version buckets with highest version 1525560876549013504
   [junit4]   2> 1590444 INFO  (coreLoadExecutor-3660-thread-1-processing-n:127.0.0.1:60037_) [n:127.0.0.1:60037_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 1590444 INFO  (coreLoadExecutor-3660-thread-1-processing-n:127.0.0.1:60037_) [n:127.0.0.1:60037_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 1590445 INFO  (coreZkRegister-3654-thread-1-processing-c:control_collection n:127.0.0.1:60037_ x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:60037_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:60037 collection:control_collection shard:shard1
   [junit4]   2> 1590445 INFO  (coreZkRegister-3654-thread-1-processing-c:control_collection n:127.0.0.1:60037_ x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:60037_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 1590448 INFO  (coreZkRegister-3654-thread-1-processing-c:control_collection n:127.0.0.1:60037_ x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:60037_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/95347554668707844-core_node1-n_0000000000
   [junit4]   2> 1590449 INFO  (coreZkRegister-3654-thread-1-processing-c:control_collection n:127.0.0.1:60037_ x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:60037_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard=shard1 and weAreReplacement=false and leaderVoteWait=180000
   [junit4]   2> 1590451 INFO  (coreZkRegister-3654-thread-1-processing-c:control_collection n:127.0.0.1:60037_ x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:60037_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1590451 INFO  (coreZkRegister-3654-thread-1-processing-c:control_collection n:127.0.0.1:60037_ x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:60037_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1590451 INFO  (coreZkRegister-3654-thread-1-processing-c:control_collection n:127.0.0.1:60037_ x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:60037_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:60037/collection1/
   [junit4]   2> 1590451 INFO  (coreZkRegister-3654-thread-1-processing-c:control_collection n:127.0.0.1:60037_ x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:60037_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 1590451 INFO  (coreZkRegister-3654-thread-1-processing-c:control_collection n:127.0.0.1:60037_ x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:60037_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:60037/collection1/ has no replicas
   [junit4]   2> 1590451 INFO  (OverseerStateUpdate-95347554668707844-127.0.0.1:60037_-n_0000000000) [n:127.0.0.1:60037_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 1590452 INFO  (coreZkRegister-3654-thread-1-processing-c:control_collection n:127.0.0.1:60037_ x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:60037_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 1590454 INFO  (coreZkRegister-3654-thread-1-processing-c:control_collection n:127.0.0.1:60037_ x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:60037_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 1590456 INFO  (coreZkRegister-3654-thread-1-processing-c:control_collection n:127.0.0.1:60037_ x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:60037_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:60037/collection1/ shard1
   [junit4]   2> 1590465 INFO  (OverseerStateUpdate-95347554668707844-127.0.0.1:60037_-n_0000000000) [n:127.0.0.1:60037_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:60037",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 1
   [junit4]   2> 1590527 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1590528 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1590529 INFO  (zkCallback-1400-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@349b20c6 name:ZooKeeperConnection Watcher:127.0.0.1:48757/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1590529 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1590529 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1590529 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1590532 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 1590532 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with stateFormat=2
   [junit4]   2> 1590532 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1590532 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1590533 INFO  (zkCallback-1401-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@58fa7c2f name:ZooKeeperConnection Watcher:127.0.0.1:48757/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1590533 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1590534 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1590536 INFO  (OverseerStateUpdate-95347554668707844-127.0.0.1:60037_-n_0000000000) [n:127.0.0.1:60037_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"create",
   [junit4]   2>   "name":"collection1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "stateFormat":"2"} current state version: 1
   [junit4]   2> 1590536 INFO  (OverseerStateUpdate-95347554668707844-127.0.0.1:60037_-n_0000000000) [n:127.0.0.1:60037_    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 1590537 INFO  (zkCallback-1398-thread-1-processing-n:127.0.0.1:60037_) [n:127.0.0.1:60037_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1590543 INFO  (zkCallback-1400-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1590566 INFO  (coreZkRegister-3654-thread-1-processing-c:control_collection n:127.0.0.1:60037_ x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:60037_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:60037/collection1/ and leader is http://127.0.0.1:60037/collection1/
   [junit4]   2> 1590566 INFO  (coreZkRegister-3654-thread-1-processing-c:control_collection n:127.0.0.1:60037_ x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:60037_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:60037
   [junit4]   2> 1590566 INFO  (coreZkRegister-3654-thread-1-processing-c:control_collection n:127.0.0.1:60037_ x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:60037_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1590567 INFO  (coreZkRegister-3654-thread-1-processing-c:control_collection n:127.0.0.1:60037_ x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:60037_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 1590567 INFO  (coreZkRegister-3654-thread-1-processing-c:control_collection n:127.0.0.1:60037_ x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:60037_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1590569 INFO  (OverseerStateUpdate-95347554668707844-127.0.0.1:60037_-n_0000000000) [n:127.0.0.1:60037_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "node_name":"127.0.0.1:60037_",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "base_url":"http://127.0.0.1:60037",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null} current state version: 2
   [junit4]   2> 1590570 INFO  (OverseerStateUpdate-95347554668707844-127.0.0.1:60037_-n_0000000000) [n:127.0.0.1:60037_    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "node_name":"127.0.0.1:60037_",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "base_url":"http://127.0.0.1:60037",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null}
   [junit4]   2> 1590571 INFO  (OverseerStateUpdate-95347554668707844-127.0.0.1:60037_-n_0000000000) [n:127.0.0.1:60037_    ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/collection1/state.json
   [junit4]   2> 1590622 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_7AF4185537C08820-001/shard-1-001/cores/collection1
   [junit4]   2> 1590623 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_7AF4185537C08820-001/shard-1-001
   [junit4]   2> 1590623 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 1590625 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@d260f3{/,null,AVAILABLE}
   [junit4]   2> 1590625 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [    ] o.e.j.s.ServerConnector Started ServerConnector@2b41fefc{HTTP/1.1}{127.0.0.1:34097}
   [junit4]   2> 1590626 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [    ] o.e.j.s.Server Started @1593291ms
   [junit4]   2> 1590626 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostPort=34097, hostContext=/, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_7AF4185537C08820-001/shard-1-001/cores, solrconfig=solrconfig.xml, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_7AF4185537C08820-001/tempDir-001/jetty1}
   [junit4]   2> 1590626 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@4f6de641
   [junit4]   2> 1590626 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_7AF4185537C08820-001/shard-1-001'
   [junit4]   2> 1590626 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1590626 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1590626 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1590627 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1590628 INFO  (zkCallback-1402-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2933886e name:ZooKeeperConnection Watcher:127.0.0.1:48757/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1590628 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1590629 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1590630 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1590630 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_7AF4185537C08820-001/shard-1-001/solr.xml
   [junit4]   2> 1590644 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_7AF4185537C08820-001/shard-1-001/cores
   [junit4]   2> 1590644 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [    ] o.a.s.c.CoreContainer New CoreContainer 1220846646
   [junit4]   2> 1590644 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_7AF4185537C08820-001/shard-1-001]
   [junit4]   2> 1590645 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_7AF4185537C08820-001/shard-1-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_7AF4185537C08820-001/shard-1-001/lib
   [junit4]   2> 1590645 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1590646 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1590647 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1590647 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1590647 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:48757/solr
   [junit4]   2> 1590647 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1590647 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1590648 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1590649 INFO  (zkCallback-1405-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@359abd4a name:ZooKeeperConnection Watcher:127.0.0.1:48757 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1590649 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1590649 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1590651 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [n:127.0.0.1:34097_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1590652 INFO  (zkCallback-1406-thread-1-processing-n:127.0.0.1:34097_) [n:127.0.0.1:34097_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@465bd76b name:ZooKeeperConnection Watcher:127.0.0.1:48757/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1590652 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [n:127.0.0.1:34097_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1590657 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [n:127.0.0.1:34097_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1590675 INFO  (zkCallback-1400-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1590675 INFO  (zkCallback-1398-thread-1-processing-n:127.0.0.1:60037_) [n:127.0.0.1:60037_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1590675 INFO  (zkCallback-1406-thread-1-processing-n:127.0.0.1:34097_) [n:127.0.0.1:34097_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1591664 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [n:127.0.0.1:34097_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:34097_
   [junit4]   2> 1591664 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [n:127.0.0.1:34097_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:34097_
   [junit4]   2> 1591665 INFO  (zkCallback-1406-thread-1-processing-n:127.0.0.1:34097_) [n:127.0.0.1:34097_    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1591665 INFO  (zkCallback-1398-thread-1-processing-n:127.0.0.1:60037_) [n:127.0.0.1:60037_    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1591666 INFO  (zkCallback-1400-thread-1) [    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1591667 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [n:127.0.0.1:34097_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1591667 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [n:127.0.0.1:34097_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/95347554668707849-127.0.0.1:34097_-n_0000000001
   [junit4]   2> 1591668 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [n:127.0.0.1:34097_    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/95347554668707844-127.0.0.1:60037_-n_0000000000 to know if I could be the leader
   [junit4]   2> 1591679 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [n:127.0.0.1:34097_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1591679 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [n:127.0.0.1:34097_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1591680 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [n:127.0.0.1:34097_    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_7AF4185537C08820-001/shard-1-001/cores
   [junit4]   2> 1591681 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [n:127.0.0.1:34097_    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {config=solrconfig.xml, coreNodeName=, shard=, loadOnStartup=true, configSetProperties=configsetprops.json, name=collection1, dataDir=data/, collection=collection1, transient=false, schema=schema.xml}
   [junit4]   2> 1591681 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [n:127.0.0.1:34097_    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_7AF4185537C08820-001/shard-1-001/cores/collection1
   [junit4]   2> 1591681 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [n:127.0.0.1:34097_    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 1591684 INFO  (coreLoadExecutor-3671-thread-1-processing-n:127.0.0.1:34097_) [n:127.0.0.1:34097_ c:collection1   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 1591684 INFO  (coreLoadExecutor-3671-thread-1-processing-n:127.0.0.1:34097_) [n:127.0.0.1:34097_ c:collection1   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1591686 INFO  (OverseerStateUpdate-95347554668707844-127.0.0.1:60037_-n_0000000000) [n:127.0.0.1:60037_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "node_name":"127.0.0.1:34097_",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:34097",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "roles":null} current state version: 3
   [junit4]   2> 1591687 INFO  (OverseerStateUpdate-95347554668707844-127.0.0.1:60037_-n_0000000000) [n:127.0.0.1:60037_    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "node_name":"127.0.0.1:34097_",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:34097",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "roles":null}
   [junit4]   2> 1591687 INFO  (OverseerStateUpdate-95347554668707844-127.0.0.1:60037_-n_0000000000) [n:127.0.0.1:60037_    ] o.a.s.c.o.ReplicaMutator Collection already exists with numShards=2
   [junit4]   2> 1591687 INFO  (OverseerStateUpdate-95347554668707844-127.0.0.1:60037_-n_0000000000) [n:127.0.0.1:60037_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 1591689 INFO  (coreLoadExecutor-3671-thread-1-processing-n:127.0.0.1:34097_) [n:127.0.0.1:34097_ c:collection1   x:collection1] o.a.s.c.ZkController Registering watch for collection collection1
   [junit4]   2> 1591689 INFO  (coreLoadExecutor-3671-thread-1-processing-n:127.0.0.1:34097_) [n:127.0.0.1:34097_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader addZkWatch [collection1]
   [junit4]   2> 1591690 INFO  (coreLoadExecutor-3671-thread-1-processing-n:127.0.0.1:34097_) [n:127.0.0.1:34097_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Add data for [collection1] ver [0]
   [junit4]   2> 1591690 INFO  (coreLoadExecutor-3671-thread-1-processing-n:127.0.0.1:34097_) [n:127.0.0.1:34097_ c:collection1   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 1591695 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [n:127.0.0.1:34097_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 1591695 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AF4185537C08820]) [n:127.0.0.1:34097_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1591789 INFO  (OverseerStateUpdate-95347554668707844-127.0.0.1:60037_-n_0000000000) [n:127.0.0.1:60037_    ] o.a.s.c.o.ZkStateWriter going to update_collection /collections/collection1/state.json version: 0
   [junit4]   2> 1591790 INFO  (zkCallback-1406-thread-1-processing-n:127.0.0.1:34097_) [n:127.0.0.1:34097_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1591790 INFO  (zkCallback-1406-thread-1-processing-n:127.0.0.1:34097_) [n:127.0.0.1:34097_    ] o.a.s.c.c.ZkStateReader Updating data for [collection1] from [0] to [1]
   [junit4]   2> 1592690 INFO  (coreLoadExecutor-3671-thread-1-processing-n:127.0.0.1:34097_) [n:127.0.0.1:34097_ c:collection1   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1592690 INFO  (coreLoadExecutor-3671-thread-1-processing-n:127.0.0.1:34097_) [n:127.0.0.1:34097_ c:collection1   x:collection1] o.a.s.c.ZkController Check for collection zkNode:collection1
   [junit4]   2> 1592691 INFO  (coreLoadExecutor-3671-thread-1-processing-n:127.0.0.1:34097_) [n:127.0.0.1:34097_ c:collection1   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 1592691 INFO  (coreLoadExecutor-3671-thread-1-processing-n:127.0.0.1:34097_) [n:127.0.0.1:34097_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/collection1]
   [junit4]   2> 1592691 INFO  (coreLoadExecutor-3671-thread-1-processing-n:127.0.0.1:34097_) [n:127.0.0.1:34097_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/collection1] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 1592691 INFO  (coreLoadExecutor-3671-thread-1-processing-n:127.0.0.1:34097_) [n:127.0.0.1:34097_ c:collection1   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_7AF4185537C08820-001/shard-1-001/cores/collection1'
   [junit4]   2> 1592691 INFO  (coreLoadExecutor-3671-thread-1-processing-n:127.0.0.1:34097_) [n:127.0.0.1:34097_ c:collection1   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1592691 INFO  (coreLoadExecutor-3671-thread-1-processing-n:127.0.0.1:34097_) [n:127.0.0.1:34097_ c:collection1   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1592693 INFO  (coreLoadExecutor-3671-thread-1-processing-n:127.0.0.1:34097_) [n:127.0.0.1:34097_ c:collection1   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 1592699 INFO  (coreLoadExecutor-3671-thread-1-processing-n:127.0.0.1:34097_) [n:127.0.0.1:34097_ c:collection1   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 1592738 INFO  (coreLoadExecutor-3671-thread-1-processing-n:127.0.0.1:34097_) [n:127.0.0.1:34097_ c:collection1   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.5.0
   [junit4]   2> 1592769 INFO  (coreLoadExecutor-3671-thread-1-processing-n:127.0.0.1:34097_) [n:127.0.0.1:34097_ c:collection1   x:collection1] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1592779 INFO  (coreLoadExecutor-3671-thread-1-processing-n:127.0.0.1:34097_) [n:127.0.0.1:34097_ c:collection1   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1593049 INFO  (coreLoadExecutor-3671-thread-1-processing-n:127.0.0.1:34097_) [n:127.0.0.1:34097_ c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 1593058 INFO  (coreLoadExecutor-3671-thread-1-processing-n:127.0.0.1:34097_) [n:127.0.0.1:34097_ c:collection1   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 1593060 INFO  (coreLoadExecutor-3671-thread-1-processing-n:127.0.0.1:34097_) [n:127.0.0.1:34097_ c:collection1   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 1593069 INFO  (coreLoadExecutor-3671-thread-1-processing-n:127.0.0.1:34097_) [n:127.0.0.1:34097_ c:collection1   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1593073 INFO  (coreLoadExecutor-3671-thread-1-processing-n:127.0.0.1:34097_) [n:127.0.0.1:34097_ c:collection1   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1593076 INFO  (coreLoadExecutor-3671-thread-1-processing-n:127.0.0.1:34097_) [n:127.0.0.1:34097_ c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1593076 INFO  (coreLoadExecutor-3671-thread-1-processing-n:127.0.0.1:34097_) [n:127.0.0.1:34097_ c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1593078 INFO  (coreLoadExecutor-3671-thread-1-processing-n:127.0.0.1:34097_) [n:127.0.0.1:34097_ c:collection1   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 1593078 INFO  (coreLoadExecutor-3671-thread-1-processing-n:127.0.0.1:34097_) [n:127.0.0.1:34097_ c:collection1   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection collection1
   [junit4]   2> 1593078 INFO  (coreLoadExecutor-3671-thread-1-processing-n:127.0.0.1:34097_) [n:127.0.0.1:34097_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 1593078 INFO  (coreLoadExecutor-3671-thread-1-processing-n:127.0.0.1:34097_) [n:127.0.0.1:34097_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_7AF4185537C08820-001/shard-1-001/cores/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_7AF4185537C08820-001/shard-1-001/cores/collection1/data/]
   [junit4]   2> 1593078 INFO  (coreLoadExecutor-36

[...truncated too long message...]

ient.execute(CloseableHttpClient.java:55)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:481)
   [junit4]    > 	... 11 moreThrowable #6: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=15772, name=collection2, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    > Caused by: java.lang.RuntimeException: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:56566, http://127.0.0.1:60037, http://127.0.0.1:34097]
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([7AF4185537C08820]:0)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:890)
   [junit4]    > Caused by: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:56566, http://127.0.0.1:60037, http://127.0.0.1:34097]
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:352)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1573)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1594)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:880)
   [junit4]    > Caused by: org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:56566
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:585)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:240)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:229)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
   [junit4]    > 	... 7 more
   [junit4]    > Caused by: java.net.SocketTimeoutException: Read timed out
   [junit4]    > 	at java.net.SocketInputStream.socketRead0(Native Method)
   [junit4]    > 	at java.net.SocketInputStream.read(SocketInputStream.java:152)
   [junit4]    > 	at java.net.SocketInputStream.read(SocketInputStream.java:122)
   [junit4]    > 	at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:160)
   [junit4]    > 	at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:84)
   [junit4]    > 	at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:273)
   [junit4]    > 	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
   [junit4]    > 	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
   [junit4]    > 	at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
   [junit4]    > 	at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)
   [junit4]    > 	at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)
   [junit4]    > 	at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197)
   [junit4]    > 	at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)
   [junit4]    > 	at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)
   [junit4]    > 	at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685)
   [junit4]    > 	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487)
   [junit4]    > 	at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
   [junit4]    > 	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
   [junit4]    > 	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
   [junit4]    > 	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:481)
   [junit4]    > 	... 11 more
   [junit4]   2> 2616951 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[7AF4185537C08820]-worker) [n:127.0.0.1:51622_ c:addReplicaColl s:shard1 r:core_node7 x:propertyDotName] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene54), sim=RandomSimilarity(queryNorm=true,coord=crazy): {}, locale=es-PR, timezone=PST8PDT
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_80 (64-bit)/cpus=4,threads=1,free=181402408,total=479723520
   [junit4]   2> NOTE: All tests run in this JVM: [MBeansHandlerTest, LeaderInitiatedRecoveryOnCommitTest, SSLMigrationTest, ChaosMonkeyNothingIsSafeTest, TestSolrConfigHandlerConcurrent, TestBlobHandler, TestQueryUtils, ForceLeaderTest, SegmentsInfoRequestHandlerTest, OverseerCollectionConfigSetProcessorTest, ZkStateReaderTest, SpellPossibilityIteratorTest, TermsComponentTest, AliasIntegrationTest, DistribJoinFromCollectionTest, TestUpdate, TestQuerySenderListener, TestQuerySenderNoQuery, TestDynamicFieldCollectionResource, TriLevelCompositeIdRoutingTest, TestSortByMinMaxFunction, ConjunctionSolrSpellCheckerTest, AnalysisErrorHandlingTest, TestSolrQueryResponse, TestCoreContainer, SchemaVersionSpecificBehaviorTest, TestStressUserVersions, SolrIndexSplitterTest, TestNoOpRegenerator, TestClassNameShortening, ExitableDirectoryReaderTest, TestStressVersions, FullHLLTest, BlockJoinFacetRandomTest, FastVectorHighlighterTest, DocValuesTest, NotRequiredUniqueKeyTest, ReplicationFactorTest, TestJsonFacets, TestSimpleTrackingShardHandler, ConvertedLegacyTest, NumericFieldsTest, SoftAutoCommitTest, ClusterStateUpdateTest, ShardRoutingCustomTest, TestNonDefinedSimilarityFactory, TestShortCircuitedRequests, TestTrackingShardHandlerFactory, HighlighterTest, TestDistributedStatsComponentCardinality, TestWordDelimiterFilterFactory, TestRemoveLastDynamicCopyField, DocumentBuilderTest, TestManagedStopFilterFactory, ZkSolrClientTest, PreAnalyzedUpdateProcessorTest, ZkCLITest, TestTrie, MinimalSchemaTest, TestExceedMaxTermLength, TestSolrDynamicMBean, TestCursorMarkWithoutUniqueKey, SuggesterFSTTest, MoreLikeThisHandlerTest, TestCollationFieldDocValues, TestUseDocValuesAsStored, SpatialHeatmapFacetsTest, OverseerTaskQueueTest, TestDFRSimilarityFactory, SpellCheckComponentTest, TestMiniSolrCloudClusterSSL, TestDistributedGrouping, DistributedMLTComponentTest, DeleteInactiveReplicaTest, SolrCmdDistributorTest, TestIntervalFaceting, FileUtilsTest, WordBreakSolrSpellCheckerTest, AsyncMigrateRouteKeyTest, TestFiltering, ChangedSchemaMergeTest, RankQueryTest, TestConfigOverlay, SolrIndexConfigTest, TestSmileRequest, TestLMJelinekMercerSimilarityFactory, SpellCheckCollatorTest, CheckHdfsIndexTest, TestExclusionRuleCollectionAccess, TestSolrDeletionPolicy2, BigEndianAscendingWordSerializerTest, TestDocumentBuilder, BasicDistributedZkTest, ChaosMonkeySafeLeaderTest, BasicDistributedZk2Test, CollectionsAPIDistributedZkTest]
   [junit4] Completed [295/566 (1!)] on J1 in 1029.18s, 1 test, 1 error <<< FAILURES!

[...truncated 860 lines...]
   [junit4] JVM J1: stdout was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J1-20160207_231032_793.sysout
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] java.lang.OutOfMemoryError: Java heap space
   [junit4] Dumping heap to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/heapdumps/java_pid8933.hprof ...
   [junit4] Heap dump file created [626566346 bytes in 12.335 secs]
   [junit4] <<< JVM J1: EOF ----

[...truncated 16 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:757: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:694: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:534: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1477: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1033: There were test failures: 566 suites (7 ignored), 2268 tests, 1 error, 33 ignored (18 assumptions) [seed: 7AF4185537C08820]

Total time: 209 minutes 52 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1090 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1090/

1 tests failed.
FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=664, name=collection1, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=664, name=collection1, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:44332/zo/ji: collection already exists: awholynewstresscollection_collection1_1
	at __randomizedtesting.SeedInfo.seed([20A35177A4CD4759]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:576)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:240)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:229)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1573)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1594)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:880)




Build Log:
[...truncated 11175 lines...]
   [junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_20A35177A4CD4759-001/init-core-data-001
   [junit4]   2> 9244 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[20A35177A4CD4759]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (false)
   [junit4]   2> 9245 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[20A35177A4CD4759]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /zo/ji
   [junit4]   2> 9289 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 9307 INFO  (Thread-18) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 9307 INFO  (Thread-18) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 9397 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [    ] o.a.s.c.ZkTestServer start zk server on port:47790
   [junit4]   2> 9437 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 9505 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 9517 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn Exception causing close of session 0x0 due to java.io.IOException: ZooKeeperServer not running
   [junit4]   2> 11608 INFO  (zkCallback-5-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@32d7f5c5 name:ZooKeeperConnection Watcher:127.0.0.1:47790 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 11608 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 11609 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 11611 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 11650 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x152b9d4a7c20000, likely client has closed socket
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 11653 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 11654 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 11656 INFO  (zkCallback-6-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5962c302 name:ZooKeeperConnection Watcher:127.0.0.1:47790/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 11657 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 11657 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 11658 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 11664 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 11668 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 11672 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 11685 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 11696 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 11703 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 11704 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 11709 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 11709 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 11716 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 11716 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 11720 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 11720 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 11724 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 11725 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 11732 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 11732 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 11736 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 11736 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 11739 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 11740 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 11744 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 11744 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 11748 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 11748 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 11761 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 11762 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 11764 INFO  (zkCallback-7-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3c6c43ae name:ZooKeeperConnection Watcher:127.0.0.1:47790/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 11764 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 11764 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 12067 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_20A35177A4CD4759-001/control-001/cores/collection1
   [junit4]   2> 12083 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 12087 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5c19b253{/zo/ji,null,AVAILABLE}
   [junit4]   2> 12322 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [    ] o.e.j.s.ServerConnector Started ServerConnector@30efd5fe{SSL-http/1.1}{127.0.0.1:34285}
   [junit4]   2> 12322 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [    ] o.e.j.s.Server Started @15051ms
   [junit4]   2> 12322 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/zo/ji, hostPort=34285, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_20A35177A4CD4759-001/control-001/cores, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_20A35177A4CD4759-001/tempDir-001/control/data}
   [junit4]   2> 12323 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@4f6de641
   [junit4]   2> 12324 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_20A35177A4CD4759-001/control-001'
   [junit4]   2> 12324 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 12324 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 12325 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 12325 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 12332 INFO  (zkCallback-8-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@20a45159 name:ZooKeeperConnection Watcher:127.0.0.1:47790/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 12333 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 12333 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 12346 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 12346 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_20A35177A4CD4759-001/control-001/solr.xml
   [junit4]   2> 12390 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_20A35177A4CD4759-001/control-001/cores
   [junit4]   2> 12390 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [    ] o.a.s.c.CoreContainer New CoreContainer 715524417
   [junit4]   2> 12390 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_20A35177A4CD4759-001/control-001]
   [junit4]   2> 12390 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_20A35177A4CD4759-001/control-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_20A35177A4CD4759-001/control-001/lib
   [junit4]   2> 12391 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 12410 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 12417 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 12418 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 12418 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:47790/solr
   [junit4]   2> 12457 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 12458 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 12479 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 12482 INFO  (zkCallback-11-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6c37bfd6 name:ZooKeeperConnection Watcher:127.0.0.1:47790 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 12482 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 12483 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 12500 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [n:127.0.0.1:34285_zo%2Fji    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 12501 INFO  (zkCallback-12-thread-1-processing-n:127.0.0.1:34285_zo%2Fji) [n:127.0.0.1:34285_zo%2Fji    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7bb39e5e name:ZooKeeperConnection Watcher:127.0.0.1:47790/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 12502 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [n:127.0.0.1:34285_zo%2Fji    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 12516 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [n:127.0.0.1:34285_zo%2Fji    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 12527 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [n:127.0.0.1:34285_zo%2Fji    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 12548 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [n:127.0.0.1:34285_zo%2Fji    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 12555 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [n:127.0.0.1:34285_zo%2Fji    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 12565 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [n:127.0.0.1:34285_zo%2Fji    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 12576 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [n:127.0.0.1:34285_zo%2Fji    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 12580 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [n:127.0.0.1:34285_zo%2Fji    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 12583 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [n:127.0.0.1:34285_zo%2Fji    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 12595 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [n:127.0.0.1:34285_zo%2Fji    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 12598 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [n:127.0.0.1:34285_zo%2Fji    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:34285_zo%2Fji
   [junit4]   2> 12601 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [n:127.0.0.1:34285_zo%2Fji    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:34285_zo%2Fji
   [junit4]   2> 12609 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [n:127.0.0.1:34285_zo%2Fji    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 12612 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [n:127.0.0.1:34285_zo%2Fji    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 12615 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [n:127.0.0.1:34285_zo%2Fji    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 12617 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [n:127.0.0.1:34285_zo%2Fji    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/95342865126916101-127.0.0.1:34285_zo%2Fji-n_0000000000
   [junit4]   2> 12630 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [n:127.0.0.1:34285_zo%2Fji    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:34285_zo%2Fji
   [junit4]   2> 12630 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [n:127.0.0.1:34285_zo%2Fji    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 12633 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [n:127.0.0.1:34285_zo%2Fji    ] o.a.s.c.Overseer Overseer (id=95342865126916101-127.0.0.1:34285_zo%2Fji-n_0000000000) starting
   [junit4]   2> 12643 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [n:127.0.0.1:34285_zo%2Fji    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 12680 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [n:127.0.0.1:34285_zo%2Fji    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 12689 INFO  (OverseerCollectionConfigSetProcessor-95342865126916101-127.0.0.1:34285_zo%2Fji-n_0000000000) [n:127.0.0.1:34285_zo%2Fji    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 12696 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [n:127.0.0.1:34285_zo%2Fji    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 12688 INFO  (OverseerStateUpdate-95342865126916101-127.0.0.1:34285_zo%2Fji-n_0000000000) [n:127.0.0.1:34285_zo%2Fji    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 12817 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [n:127.0.0.1:34285_zo%2Fji    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 12817 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [n:127.0.0.1:34285_zo%2Fji    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 12819 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [n:127.0.0.1:34285_zo%2Fji    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_20A35177A4CD4759-001/control-001/cores
   [junit4]   2> 12822 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [n:127.0.0.1:34285_zo%2Fji    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {schema=schema.xml, loadOnStartup=true, collection=control_collection, config=solrconfig.xml, coreNodeName=, shard=, configSetProperties=configsetprops.json, name=collection1, dataDir=data/, transient=false}
   [junit4]   2> 12822 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [n:127.0.0.1:34285_zo%2Fji    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_20A35177A4CD4759-001/control-001/cores/collection1
   [junit4]   2> 12823 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [n:127.0.0.1:34285_zo%2Fji    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 12827 INFO  (coreLoadExecutor-28-thread-1-processing-n:127.0.0.1:34285_zo%2Fji) [n:127.0.0.1:34285_zo%2Fji c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 12827 INFO  (coreLoadExecutor-28-thread-1-processing-n:127.0.0.1:34285_zo%2Fji) [n:127.0.0.1:34285_zo%2Fji c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 12855 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [n:127.0.0.1:34285_zo%2Fji    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 12855 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [n:127.0.0.1:34285_zo%2Fji    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 12858 INFO  (OverseerStateUpdate-95342865126916101-127.0.0.1:34285_zo%2Fji-n_0000000000) [n:127.0.0.1:34285_zo%2Fji    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "base_url":"https://127.0.0.1:34285/zo/ji",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "node_name":"127.0.0.1:34285_zo%2Fji",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "state":"down",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 12866 INFO  (coreLoadExecutor-28-thread-1-processing-n:127.0.0.1:34285_zo%2Fji) [n:127.0.0.1:34285_zo%2Fji c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 12871 INFO  (OverseerStateUpdate-95342865126916101-127.0.0.1:34285_zo%2Fji-n_0000000000) [n:127.0.0.1:34285_zo%2Fji    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "base_url":"https://127.0.0.1:34285/zo/ji",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "node_name":"127.0.0.1:34285_zo%2Fji",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "state":"down",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 12881 INFO  (OverseerStateUpdate-95342865126916101-127.0.0.1:34285_zo%2Fji-n_0000000000) [n:127.0.0.1:34285_zo%2Fji    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 12895 INFO  (OverseerStateUpdate-95342865126916101-127.0.0.1:34285_zo%2Fji-n_0000000000) [n:127.0.0.1:34285_zo%2Fji    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 12904 INFO  (zkCallback-12-thread-1-processing-n:127.0.0.1:34285_zo%2Fji) [n:127.0.0.1:34285_zo%2Fji    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 13867 INFO  (coreLoadExecutor-28-thread-1-processing-n:127.0.0.1:34285_zo%2Fji) [n:127.0.0.1:34285_zo%2Fji c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 13867 INFO  (coreLoadExecutor-28-thread-1-processing-n:127.0.0.1:34285_zo%2Fji) [n:127.0.0.1:34285_zo%2Fji c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 13868 INFO  (coreLoadExecutor-28-thread-1-processing-n:127.0.0.1:34285_zo%2Fji) [n:127.0.0.1:34285_zo%2Fji c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 13868 INFO  (coreLoadExecutor-28-thread-1-processing-n:127.0.0.1:34285_zo%2Fji) [n:127.0.0.1:34285_zo%2Fji c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/control_collection]
   [junit4]   2> 13870 INFO  (coreLoadExecutor-28-thread-1-processing-n:127.0.0.1:34285_zo%2Fji) [n:127.0.0.1:34285_zo%2Fji c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/control_collection] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 13870 INFO  (coreLoadExecutor-28-thread-1-processing-n:127.0.0.1:34285_zo%2Fji) [n:127.0.0.1:34285_zo%2Fji c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_20A35177A4CD4759-001/control-001/cores/collection1'
   [junit4]   2> 13870 INFO  (coreLoadExecutor-28-thread-1-processing-n:127.0.0.1:34285_zo%2Fji) [n:127.0.0.1:34285_zo%2Fji c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 13871 INFO  (coreLoadExecutor-28-thread-1-processing-n:127.0.0.1:34285_zo%2Fji) [n:127.0.0.1:34285_zo%2Fji c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 13874 INFO  (coreLoadExecutor-28-thread-1-processing-n:127.0.0.1:34285_zo%2Fji) [n:127.0.0.1:34285_zo%2Fji c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 13895 INFO  (coreLoadExecutor-28-thread-1-processing-n:127.0.0.1:34285_zo%2Fji) [n:127.0.0.1:34285_zo%2Fji c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 13917 INFO  (coreLoadExecutor-28-thread-1-processing-n:127.0.0.1:34285_zo%2Fji) [n:127.0.0.1:34285_zo%2Fji c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.5.0
   [junit4]   2> 13985 INFO  (coreLoadExecutor-28-thread-1-processing-n:127.0.0.1:34285_zo%2Fji) [n:127.0.0.1:34285_zo%2Fji c:control_collection   x:collection1] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 14062 INFO  (coreLoadExecutor-28-thread-1-processing-n:127.0.0.1:34285_zo%2Fji) [n:127.0.0.1:34285_zo%2Fji c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 14459 INFO  (coreLoadExecutor-28-thread-1-processing-n:127.0.0.1:34285_zo%2Fji) [n:127.0.0.1:34285_zo%2Fji c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 14477 INFO  (coreLoadExecutor-28-thread-1-processing-n:127.0.0.1:34285_zo%2Fji) [n:127.0.0.1:34285_zo%2Fji c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 14479 INFO  (coreLoadExecutor-28-thread-1-processing-n:127.0.0.1:34285_zo%2Fji) [n:127.0.0.1:34285_zo%2Fji c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 14522 INFO  (coreLoadExecutor-28-thread-1-processing-n:127.0.0.1:34285_zo%2Fji) [n:127.0.0.1:34285_zo%2Fji c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 14530 INFO  (coreLoadExecutor-28-thread-1-processing-n:127.0.0.1:34285_zo%2Fji) [n:127.0.0.1:34285_zo%2Fji c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 14536 INFO  (coreLoadExecutor-28-thread-1-processing-n:127.0.0.1:34285_zo%2Fji) [n:127.0.0.1:34285_zo%2Fji c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 14537 INFO  (coreLoadExecutor-28-thread-1-processing-n:127.0.0.1:34285_zo%2Fji) [n:127.0.0.1:34285_zo%2Fji c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 14540 INFO  (coreLoadExecutor-28-thread-1-processing-n:127.0.0.1:34285_zo%2Fji) [n:127.0.0.1:34285_zo%2Fji c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 14540 INFO  (coreLoadExecutor-28-thread-1-processing-n:127.0.0.1:34285_zo%2Fji) [n:127.0.0.1:34285_zo%2Fji c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 14541 INFO  (coreLoadExecutor-28-thread-1-processing-n:127.0.0.1:34285_zo%2Fji) [n:127.0.0.1:34285_zo%2Fji c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 14541 INFO  (coreLoadExecutor-28-thread-1-processing-n:127.0.0.1:34285_zo%2Fji) [n:127.0.0.1:34285_zo%2Fji c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_20A35177A4CD4759-001/control-001/cores/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_20A35177A4CD4759-001/control-001/cores/collection1/data/]
   [junit4]   2> 14542 INFO  (coreLoadExecutor-28-thread-1-processing-n:127.0.0.1:34285_zo%2Fji) [n:127.0.0.1:34285_zo%2Fji c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@6a309b87
   [junit4]   2> 14564 INFO  (coreLoadExecutor-28-thread-1-processing-n:127.0.0.1:34285_zo%2Fji) [n:127.0.0.1:34285_zo%2Fji c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_20A35177A4CD4759-001/control-001/cores/collection1/data
   [junit4]   2> 14564 INFO  (coreLoadExecutor-28-thread-1-processing-n:127.0.0.1:34285_zo%2Fji) [n:127.0.0.1:34285_zo%2Fji c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_20A35177A4CD4759-001/control-001/cores/collection1/data/index/
   [junit4]   2> 14565 WARN  (coreLoadExecutor-28-thread-1-processing-n:127.0.0.1:34285_zo%2Fji) [n:127.0.0.1:34285_zo%2Fji c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_20A35177A4CD4759-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 14565 INFO  (coreLoadExecutor-28-thread-1-processing-n:127.0.0.1:34285_zo%2Fji) [n:127.0.0.1:34285_zo%2Fji c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_20A35177A4CD4759-001/control-001/cores/collection1/data/index
   [junit4]   2> 14566 INFO  (coreLoadExecutor-28-thread-1-processing-n:127.0.0.1:34285_zo%2Fji) [n:127.0.0.1:34285_zo%2Fji c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=614165572, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
   [junit4]   2> 14566 INFO  (coreLoadExecutor-28-thread-1-processing-n:127.0.0.1:34285_zo%2Fji) [n:127.0.0.1:34285_zo%2Fji c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=RawDirectoryWrapper(RAMDirectory@260568a2 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2d18772e),segFN=segments_1,generation=1}
   [junit4]   2> 14566 INFO  (coreLoadExecutor-28-thread-1-processing-n:127.0.0.1:34285_zo%2Fji) [n:127.0.0.1:34285_zo%2Fji c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 14573 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:34285_zo%2Fji c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_20A35177A4CD4759-001/control-001/cores/collection1/data/
   [junit4]   2> 14588 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:34285_zo%2Fji c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_20A35177A4CD4759-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 14592 INFO  (coreLoadExecutor-28-thread-1-processing-n:127.0.0.1:34285_zo%2Fji) [n:127.0.0.1:34285_zo%2Fji c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 14592 INFO  (coreLoadExecutor-28-thread-1-processing-n:127.0.0.1:34285_zo%2Fji) [n:127.0.0.1:34285_zo%2Fji c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 14592 INFO  (coreLoadExecutor-28-thread-1-processing-n:127.0.0.1:34285_zo%2Fji) [n:127.0.0.1:34285_zo%2Fji c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 14592 INFO  (coreLoadExecutor-28-thread-1-processing-n:127.0.0.1:34285_zo%2Fji) [n:127.0.0.1:34285_zo%2Fji c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 14593 INFO  (coreLoadExecutor-28-thread-1-processing-n:127.0.0.1:34285_zo%2Fji) [n:127.0.0.1:34285_zo%2Fji c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 14593 INFO  (coreLoadExecutor-28-thread-1-processing-n:127.0.0.1:34285_zo%2Fji) [n:127.0.0.1:34285_zo%2Fji c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 14594 INFO  (coreLoadExecutor-28-thread-1-processing-n:127.0.0.1:34285_zo%2Fji) [n:127.0.0.1:34285_zo%2Fji c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 14594 INFO  (coreLoadExecutor-28-thread-1-processing-n:127.0.0.1:34285_zo%2Fji) [n:127.0.0.1:34285_zo%2Fji c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 14594 INFO  (coreLoadExecutor-28-thread-1-processing-n:127.0.0.1:34285_zo%2Fji) [n:127.0.0.1:34285_zo%2Fji c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 14595 WARN  (coreLoadExecutor-28-thread-1-processing-n:127.0.0.1:34285_zo%2Fji) [n:127.0.0.1:34285_zo%2Fji c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
   [junit4]   2> 14598 INFO  (coreLoadExecutor-28-thread-1-processing-n:127.0.0.1:34285_zo%2Fji) [n:127.0.0.1:34285_zo%2Fji c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 14602 INFO  (coreLoadExecutor-28-thread-1-processing-n:127.0.0.1:34285_zo%2Fji) [n:127.0.0.1:34285_zo%2Fji c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 14604 INFO  (coreLoadExecutor-28-thread-1-processing-n:127.0.0.1:34285_zo%2Fji) [n:127.0.0.1:34285_zo%2Fji c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 14607 INFO  (coreLoadExecutor-28-thread-1-processing-n:127.0.0.1:34285_zo%2Fji) [n:127.0.0.1:34285_zo%2Fji c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 14636 INFO  (coreLoadExecutor-28-thread-1-processing-n:127.0.0.1:34285_zo%2Fji) [n:127.0.0.1:34285_zo%2Fji c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /update/json,/admin/plugins,/update,/admin/ping,/admin/file,/admin/logging,/admin/segments,/schema,/update/csv,/dump,/admin/threads,/admin/system,/get,standard,/admin/mbeans,/config,/admin/luke,/update/json/docs,/replication,/admin/properties
   [junit4]   2> 14639 INFO  (coreLoadExecutor-28-thread-1-processing-n:127.0.0.1:34285_zo%2Fji) [n:127.0.0.1:34285_zo%2Fji c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 14640 INFO  (coreLoadExecutor-28-thread-1-processing-n:127.0.0.1:34285_zo%2Fji) [n:127.0.0.1:34285_zo%2Fji c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 14640 INFO  (coreLoadExecutor-28-thread-1-processing-n:127.0.0.1:34285_zo%2Fji) [n:127.0.0.1:34285_zo%2Fji c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 14653 INFO  (coreLoadExecutor-28-thread-1-processing-n:127.0.0.1:34285_zo%2Fji) [n:127.0.0.1:34285_zo%2Fji c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 14653 INFO  (coreLoadExecutor-28-thread-1-processing-n:127.0.0.1:34285_zo%2Fji) [n:127.0.0.1:34285_zo%2Fji c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 14657 INFO  (coreLoadExecutor-28-thread-1-processing-n:127.0.0.1:34285_zo%2Fji) [n:127.0.0.1:34285_zo%2Fji c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=31, maxMergeAtOnceExplicit=32, maxMergedSegmentMB=1.1201171875, floorSegmentMB=1.80859375, forceMergeDeletesPctAllowed=15.691105397730757, segmentsPerTier=44.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.7437868064502137
   [junit4]   2> 14657 INFO  (coreLoadExecutor-28-thread-1-processing-n:127.0.0.1:34285_zo%2Fji) [n:127.0.0.1:34285_zo%2Fji c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=RawDirectoryWrapper(RAMDirectory@260568a2 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2d18772e),segFN=segments_1,generation=1}
   [junit4]   2> 14657 INFO  (coreLoadExecutor-28-thread-1-processing-n:127.0.0.1:34285_zo%2Fji) [n:127.0.0.1:34285_zo%2Fji c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 14658 INFO  (coreLoadExecutor-28-thread-1-processing-n:127.0.0.1:34285_zo%2Fji) [n:127.0.0.1:34285_zo%2Fji c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@3f0eca9f[collection1] main
   [junit4]   2> 14658 INFO  (coreLoadExecutor-28-thread-1-processing-n:127.0.0.1:34285_zo%2Fji) [n:127.0.0.1:34285_zo%2Fji c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/control_collection]
   [junit4]   2> 14660 INFO  (coreLoadExecutor-28-thread-1-processing-n:127.0.0.1:34285_zo%2Fji) [n:127.0.0.1:34285_zo%2Fji c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/control_collection] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 14660 INFO  (coreLoadExecutor-28-thread-1-processing-n:127.0.0.1:34285_zo%2Fji) [n:127.0.0.1:34285_zo%2Fji c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 14663 INFO  (coreLoadExecutor-28-thread-1-processing-n:127.0.0.1:34285_zo%2Fji) [n:127.0.0.1:34285_zo%2Fji c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 14663 INFO  (coreLoadExecutor-28-thread-1-processing-n:127.0.0.1:34285_zo%2Fji) [n:127.0.0.1:34285_zo%2Fji c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 14664 INFO  (coreLoadExecutor-28-thread-1-processing-n:127.0.0.1:34285_zo%2Fji) [n:127.0.0.1:34285_zo%2Fji c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 14665 INFO  (coreLoadExecutor-28-thread-1-processing-n:127.0.0.1:34285_zo%2Fji) [n:127.0.0.1:34285_zo%2Fji c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 14665 INFO  (coreLoadExecutor-28-thread-1-processing-n:127.0.0.1:34285_zo%2Fji) [n:127.0.0.1:34285_zo%2Fji c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 14665 INFO  (coreLoadExecutor-28-thread-1-processing-n:127.0.0.1:34285_zo%2Fji) [n:127.0.0.1:34285_zo%2Fji c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 14665 INFO  (coreLoadExecutor-28-thread-1-processing-n:127.0.0.1:34285_zo%2Fji) [n:127.0.0.1:34285_zo%2Fji c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 14671 INFO  (searcherExecutor-29-thread-1-processing-x:collection1 c:control_collection n:127.0.0.1:34285_zo%2Fji s:shard1 r:core_node1) [n:127.0.0.1:34285_zo%2Fji c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@3f0eca9f[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 14683 INFO  (coreLoadExecutor-28-thread-1-processing-n:127.0.0.1:34285_zo%2Fji) [n:127.0.0.1:34285_zo%2Fji c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 14684 INFO  (coreLoadExecutor-28-thread-1-processing-n:127.0.0.1:34285_zo%2Fji) [n:127.0.0.1:34285_zo%2Fji c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 14684 INFO  (coreLoadExecutor-28-thread-1-processing-n:127.0.0.1:34285_zo%2Fji) [n:127.0.0.1:34285_zo%2Fji c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 14684 INFO  (coreLoadExecutor-28-thread-1-processing-n:127.0.0.1:34285_zo%2Fji) [n:127.0.0.1:34285_zo%2Fji c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1525485847559798784
   [junit4]   2> 14689 INFO  (coreLoadExecutor-28-thread-1-processing-n:127.0.0.1:34285_zo%2Fji) [n:127.0.0.1:34285_zo%2Fji c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 5.0ms to seed version buckets with highest version 1525485847559798784
   [junit4]   2> 14690 INFO  (coreLoadExecutor-28-thread-1-processing-n:127.0.0.1:34285_zo%2Fji) [n:127.0.0.1:34285_zo%2Fji c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 14694 INFO  (coreLoadExecutor-28-thread-1-processing-n:127.0.0.1:34285_zo%2Fji) [n:127.0.0.1:34285_zo%2Fji c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 14708 INFO  (coreZkRegister-22-thread-1-processing-x:collection1 c:control_collection n:127.0.0.1:34285_zo%2Fji s:shard1 r:core_node1) [n:127.0.0.1:34285_zo%2Fji c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:https://127.0.0.1:34285/zo/ji collection:control_collection shard:shard1
   [junit4]   2> 14720 INFO  (coreZkRegister-22-thread-1-processing-x:collection1 c:control_collection n:127.0.0.1:34285_zo%2Fji s:shard1 r:core_node1) [n:127.0.0.1:34285_zo%2Fji c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 14732 INFO  (coreZkRegister-22-thread-1-processing-x:collection1 c:control_collection n:127.0.0.1:34285_zo%2Fji s:shard1 r:core_node1) [n:127.0.0.1:34285_zo%2Fji c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/95342865126916101-core_node1-n_0000000000
   [junit4]   2> 14733 INFO  (coreZkRegister-22-thread-1-processing-x:collection1 c:control_collection n:127.0.0.1:34285_zo%2Fji s:shard1 r:core_node1) [n:127.0.0.1:34285_zo%2Fji c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard=shard1 and weAreReplacement=false and leaderVoteWait=180000
   [junit4]   2> 14737 INFO  (coreZkRegister-22-thread-1-processing-x:collection1 c:control_collection n:127.0.0.1:34285_zo%2Fji s:shard1 r:core_node1) [n:127.0.0.1:34285_zo%2Fji c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 14737 INFO  (coreZkRegister-22-thread-1-processing-x:collection1 c:control_collection n:127.0.0.1:34285_zo%2Fji s:shard1 r:core_node1) [n:127.0.0.1:34285_zo%2Fji c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 14737 INFO  (OverseerStateUpdate-95342865126916101-127.0.0.1:34285_zo%2Fji-n_0000000000) [n:127.0.0.1:34285_zo%2Fji    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 14738 INFO  (coreZkRegister-22-thread-1-processing-x:collection1 c:control_collection n:127.0.0.1:34285_zo%2Fji s:shard1 r:core_node1) [n:127.0.0.1:34285_zo%2Fji c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:34285/zo/ji/collection1/
   [junit4]   2> 14739 INFO  (coreZkRegister-22-thread-1-processing-x:collection1 c:control_collection n:127.0.0.1:34285_zo%2Fji s:shard1 r:core_node1) [n:127.0.0.1:34285_zo%2Fji c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 14739 INFO  (coreZkRegister-22-thread-1-processing-x:collection1 c:control_collection n:127.0.0.1:34285_zo%2Fji s:shard1 r:core_node1) [n:127.0.0.1:34285_zo%2Fji c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy https://127.0.0.1:34285/zo/ji/collection1/ has no replicas
   [junit4]   2> 14742 INFO  (zkCallback-12-thread-1-processing-n:127.0.0.1:34285_zo%2Fji) [n:127.0.0.1:34285_zo%2Fji    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 14746 INFO  (coreZkRegister-22-thread-1-processing-x:collection1 c:control_collection n:127.0.0.1:34285_zo%2Fji s:shard1 r:core_node1) [n:127.0.0.1:34285_zo%2Fji c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 14754 INFO  (coreZkRegister-22-thread-1-processing-x:collection1 c:control_collection n:127.0.0.1:34285_zo%2Fji s:shard1 r:core_node1) [n:127.0.0.1:34285_zo%2Fji c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 14777 INFO  (coreZkRegister-22-thread-1-processing-x:collection1 c:control_collection n:127.0.0.1:34285_zo%2Fji s:shard1 r:core_node1) [n:127.0.0.1:34285_zo%2Fji c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:34285/zo/ji/collection1/ shard1
   [junit4]   2> 14780 INFO  (OverseerStateUpdate-95342865126916101-127.0.0.1:34285_zo%2Fji-n_0000000000) [n:127.0.0.1:34285_zo%2Fji    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"https://127.0.0.1:34285/zo/ji",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 2
   [junit4]   2> 14895 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 14899 INFO  (zkCallback-12-thread-1-processing-n:127.0.0.1:34285_zo%2Fji) [n:127.0.0.1:34285_zo%2Fji    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 14920 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 14928 INFO  (zkCallback-14-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2d31b495 name:ZooKeeperConnection Watcher:127.0.0.1:47790/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 14928 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 14928 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 14929 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 14931 INFO  (coreZkRegister-22-thread-1-processing-x:collection1 c:control_collection n:127.0.0.1:34285_zo%2Fji s:shard1 r:core_node1) [n:127.0.0.1:34285_zo%2Fji c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are https://127.0.0.1:34285/zo/ji/collection1/ and leader is https://127.0.0.1:34285/zo/ji/collection1/
   [junit4]   2> 14931 INFO  (coreZkRegister-22-thread-1-processing-x:collection1 c:control_collection n:127.0.0.1:34285_zo%2Fji s:shard1 r:core_node1) [n:127.0.0.1:34285_zo%2Fji c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:34285/zo/ji
   [junit4]   2> 14931 INFO  (coreZkRegister-22-thread-1-processing-x:collection1 c:control_collection n:127.0.0.1:34285_zo%2Fji s:shard1 r:core_node1) [n:127.0.0.1:34285_zo%2Fji c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 14932 INFO  (coreZkRegister-22-thread-1-processing-x:collection1 c:control_collection n:127.0.0.1:34285_zo%2Fji s:shard1 r:core_node1) [n:127.0.0.1:34285_zo%2Fji c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 14932 INFO  (coreZkRegister-22-thread-1-processing-x:collection1 c:control_collection n:127.0.0.1:34285_zo%2Fji s:shard1 r:core_node1) [n:127.0.0.1:34285_zo%2Fji c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 14937 INFO  (OverseerStateUpdate-95342865126916101-127.0.0.1:34285_zo%2Fji-n_0000000000) [n:127.0.0.1:34285_zo%2Fji    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "base_url":"https://127.0.0.1:34285/zo/ji",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "node_name":"127.0.0.1:34285_zo%2Fji",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "state":"active",
   [junit4]   2>   "operation":"state"} current state version: 3
   [junit4]   2> 14939 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 14942 INFO  (OverseerStateUpdate-95342865126916101-127.0.0.1:34285_zo%2Fji-n_0000000000) [n:127.0.0.1:34285_zo%2Fji    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "base_url":"https://127.0.0.1:34285/zo/ji",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "node_name":"127.0.0.1:34285_zo%2Fji",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "state":"active",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 15056 INFO  (zkCallback-12-thread-1-processing-n:127.0.0.1:34285_zo%2Fji) [n:127.0.0.1:34285_zo%2Fji    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 15056 INFO  (zkCallback-14-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 15111 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_20A35177A4CD4759-001/shard-1-001/cores/collection1
   [junit4]   2> 15112 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_20A35177A4CD4759-001/shard-1-001
   [junit4]   2> 15113 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 15121 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@722feb6d{/zo/ji,null,AVAILABLE}
   [junit4]   2> 15156 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [    ] o.e.j.s.ServerConnector Started ServerConnector@7dfe96ab{SSL-http/1.1}{127.0.0.1:59039}
   [junit4]   2> 15156 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [    ] o.e.j.s.Server Started @17885ms
   [junit4]   2> 15157 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostPort=59039, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_20A35177A4CD4759-001/shard-1-001/cores, solrconfig=solrconfig.xml, hostContext=/zo/ji, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_20A35177A4CD4759-001/tempDir-001/jetty1}
   [junit4]   2> 15158 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@4f6de641
   [junit4]   2> 15158 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_20A35177A4CD4759-001/shard-1-001'
   [junit4]   2> 15158 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 15158 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 15159 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 15162 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 15163 INFO  (zkCallback-15-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2257a0 name:ZooKeeperConnection Watcher:127.0.0.1:47790/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 15163 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 15164 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 15166 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 15167 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_20A35177A4CD4759-001/shard-1-001/solr.xml
   [junit4]   2> 15186 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_20A35177A4CD4759-001/shard-1-001/cores
   [junit4]   2> 15187 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [    ] o.a.s.c.CoreContainer New CoreContainer 2080879949
   [junit4]   2> 15187 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_20A35177A4CD4759-001/shard-1-001]
   [junit4]   2> 15187 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_20A35177A4CD4759-001/shard-1-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_20A35177A4CD4759-001/shard-1-001/lib
   [junit4]   2> 15187 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 15191 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 15194 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 15194 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 15194 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:47790/solr
   [junit4]   2> 15194 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 15194 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 15195 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 15197 INFO  (zkCallback-18-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@55ed2e0b name:ZooKeeperConnection Watcher:127.0.0.1:47790 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 15198 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 15198 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 15202 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [n:127.0.0.1:59039_zo%2Fji    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 15213 INFO  (zkCallback-19-thread-1-processing-n:127.0.0.1:59039_zo%2Fji) [n:127.0.0.1:59039_zo%2Fji    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3bcc95d5 name:ZooKeeperConnection Watcher:127.0.0.1:47790/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 15213 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [n:127.0.0.1:59039_zo%2Fji    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 15230 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [n:127.0.0.1:59039_zo%2Fji    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 16244 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [n:127.0.0.1:59039_zo%2Fji    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:59039_zo%2Fji
   [junit4]   2> 16244 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [n:127.0.0.1:59039_zo%2Fji    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:59039_zo%2Fji
   [junit4]   2> 16247 INFO  (zkCallback-14-thread-1) [    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 16247 INFO  (zkCallback-19-thread-1-processing-n:127.0.0.1:59039_zo%2Fji) [n:127.0.0.1:59039_zo%2Fji    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 16247 INFO  (zkCallback-12-thread-1-processing-n:127.0.0.1:34285_zo%2Fji) [n:127.0.0.1:34285_zo%2Fji    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 16250 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [n:127.0.0.1:59039_zo%2Fji    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 16251 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [n:127.0.0.1:59039_zo%2Fji    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/95342865126916105-127.0.0.1:59039_zo%2Fji-n_0000000001
   [junit4]   2> 16254 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [n:127.0.0.1:59039_zo%2Fji    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/95342865126916101-127.0.0.1:34285_zo%2Fji-n_0000000000 to know if I could be the leader
   [junit4]   2> 16311 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [n:127.0.0.1:59039_zo%2Fji    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 16311 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [n:127.0.0.1:59039_zo%2Fji    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 16312 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [n:127.0.0.1:59039_zo%2Fji    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_20A35177A4CD4759-001/shard-1-001/cores
   [junit4]   2> 16314 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [n:127.0.0.1:59039_zo%2Fji    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {collection=collection1, name=collection1, loadOnStartup=true, config=solrconfig.xml, transient=false, schema=schema.xml, shard=, coreNodeName=, configSetProperties=configsetprops.json, dataDir=data/}
   [junit4]   2> 16314 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [n:127.0.0.1:59039_zo%2Fji    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_20A35177A4CD4759-001/shard-1-001/cores/collection1
   [junit4]   2> 16314 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [n:127.0.0.1:59039_zo%2Fji    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 16344 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [n:127.0.0.1:59039_zo%2Fji    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 16344 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[20A35177A4CD4759]) [n:127.0.0.1:59039_zo%2Fji    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 16345 INFO  (coreLoadExecutor-39-thread-1-processing-n:127.0.0.1:59039_zo%2Fji) [n:127.0.0.1:59039_zo%2Fji c:collection1   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 16345 INFO  (coreLoadExecutor-39-thread-1-processing-n:127.0.0.1:59039_zo%2Fji) [n:127.0.0.1:59039_zo%2Fji c:collection1   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 16349 INFO  (coreLoadExecutor-39-thread-1-processing-n:127.0.0.1:59039_zo%2Fji) [n:127.0.0.1:59039_zo%2Fji c:collection1   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 16349 INFO  (OverseerStateUpdate-95342865126916101-127.0.0.1:34285_zo%2Fji-n_0000000000) [n:127.0.0.1:34285_zo%2Fji    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "base_url":"https://127.0.0.1:59039/zo/ji",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "node_name":"127.0.0.1:59039_zo%2Fji",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "state":"down",
   [junit4]   2>   "operation":"state"} current state version: 4
   [junit4]   2> 16351 INFO  (OverseerStateUpdate-95342865126916101-127.0.0.1:34285_zo%2Fji-n_0000000000) [n:127.0.0.1:34285_zo%2Fji    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "base_url":"https://127.0.0.1:59039/zo/ji",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "node_name":"127.0.0.1:59039_zo%2Fji",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "state":"down",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 16351 INFO  (OverseerStateUpdate-95342865126916101-127.0.0.1:34285_zo%2Fji-n_0000000000) [n:127.0.0.1:34285_zo%2Fji    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 16351 INFO  (OverseerStateUpdate-95342865126916101-127.0.0.1:34285_zo%2Fji-n_0000000000) [n:127.0.0.1:34285_zo%2Fji    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 16457 INFO  (zkCallback-14-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [2])
   [junit4]   2> 16457 INFO  (zkCallback-12-thread-1-processing-n:127.0.0.1:34285_zo%2Fji) [n:127.0.0.1:34285_zo%2Fji    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [2])
   [junit4]   2> 16457 INFO  (zkCallback-19-thread-1-processing-n:127.0.0.1:59039_zo%2Fji) [n:127.0.0.1:59039_zo%2Fji    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [2])
   [junit4]   2> 17350 INFO  (coreLoadExecutor-39-thread-1-processing-n:127.0.0.1:59039_zo%2Fji) [n:127.0.0.1:59039_zo%2Fji c:collection1   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 17350 INFO  (coreLoadExecutor-39-thread-1-processing-n:127.0.0.1:59039_zo%2Fji) [n:127.0.0.1:59039_zo%2Fji c:collection1   x:collection1] o.a.s.c.ZkController Check for collection zkNode:collection1
   [junit4]   2> 17351 INFO  (coreLoadExecutor-39-thread-1-processing-n:127.0.0.1:59039_zo%2Fji) [n:127.0.0.1:59039_zo%2Fji c:collection1   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 17351 INFO  (coreLoadExecutor-39-thread-1-processing-n:127.0.0.1:59039_zo%2Fji) [n:127.0.0.1:59039_zo%2Fji c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/collection1]
   [junit4]   2> 17352 INFO  (coreLoadExecutor-39-thread-1-processing-n:127.0.0.1:59039_zo%2Fji) [n:127.0.0.1:59039_zo%2Fji c:collection1   x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/collection1] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 17352 INFO  (coreLoadExecutor-39-thread-1-processing-n:127.0.0.1:59039_zo%2Fji) [n:127.0.0.1:59039_zo%2Fji c:collection1   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_20A35177A4CD4759-001/shard-1-001/cores/collection1'
   [junit4]   2> 17353 INFO  (coreLoadExecutor-39-thread-1-processing-n:127.0.0.1:59039_zo%2Fji) [n:127.0.0.1:59039_zo%2Fji c:collection1   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 17353 INFO  (coreLoadExecutor-39-thread-1-processing-n:127.0.0.1:59039_zo%2Fji) [n:127.0.0.1:59039_zo%2Fji c:collection1   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 17355 INFO  (coreLoadExecutor-39-thread-1-processing-n:127.0.0.1:59039_zo%2Fji) [n:127.0.0.1:59039_zo%2Fji c:collection1   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 17375 INFO  (coreLoadExecutor-39-thread-1-processing-n:127.0.0.1:59039_zo%2Fji) [n:127.0.0.1:59039_zo%2Fji c:collection1   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 17395 INFO  (coreLoadExecutor-39-thread-1-processing-n:127.0.0.1:59039_zo%2Fji) [n:127.0.0.1:59039_zo%2Fji c:collection1   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.5.0
   [junit4]   2> 17439 INFO  (coreLoadExecutor-39-thread-1-processing-n:127.0.0.1:59039_zo%2Fji) [n:127.0.0.1:59039_zo%2Fji c:collection1   x:collection1] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 17534 INFO  (coreLoadExecutor-39-thread-1-processing-n:127.0.0.1:59039_zo%2Fji) [n:127.0.0.1:59039_zo%2Fji c:collection1   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 17877 INFO  (coreLoadExecutor-39-thread-1-processing-n:127.0.0.1:59039_zo%2Fji) [n:127.0.0.1:59039_zo%2Fji c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 17893 INFO  (coreLoadExecutor-39-thread-1-processing-n:127.0.0.1:59039_zo%2Fji) [n:127.0.0.1:59039_zo%2Fji c:collection1   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 17896 INFO  (coreLoadExecutor-39-thread-1-processing-n:127.0.0.1:59039_zo%2Fji) [n:127.0.0.1:59039_zo%2Fji c:collection1   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 17927 INFO  (coreLoadExecutor-39-thread-1-processing-n:127.0.0.1:59039_zo%2Fji) [n:127.0.0.1:59039_zo%2Fji c:collection1   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 17932 INFO  (coreLoadExecutor-39-thread-1-processing-n:127.0.0.1:59039_zo%2Fji) [n:127.0.0.1:59039_zo%2Fji c:collection1   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 17937 INFO  (coreLoadExecutor-39-thread-1-processing-n:127.0.0.1:59039_zo%2Fji) [n:127.0.0.1:59039_zo%2Fji c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 17938 INFO  (coreLoadExecutor-39-thread-1-processing-n:127.0.0.1:59039_zo%2Fji) [n:127.0.0.1:59039_zo%2Fji c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 17941 INFO  (coreLoadExecutor-39-thread-1-processing-n:127.0.0.1:59039_zo%2Fji) [n:127.0.0.1:59039_zo%2Fji c:collection1   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 17941 INFO  (coreLoadExecutor-39-thread-1-processing-n:127.0.0.1:59039_zo%2Fji) [n:127.0.0.1:59039_zo%2Fji c:collection1   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection collection1
   [junit4]   2> 17941 INFO  (coreLoadExecutor-39-thread-1-processing-n:127.0.0.1:59039_zo%2Fji) [n:127.0.0.1:59039_zo%2Fji c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 17942 INFO  (coreLoadExecutor-39-thread-1-processing-n:127.0.0.1:59039_zo%2Fji) [n:127.0.0.1:59039_zo%2Fji c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_20A35177A4CD4759-001/shard-1-001/cores/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_20A35177A4CD4759-001/shard-1-001/cores/collection1/data/]
   [junit4]   2> 17942 INFO  (coreLoadExecutor-39-thread-1-processing-n:127.0.0.1:59039_zo%2Fji) [n:127.0.0.1:59039_zo%2Fji c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@6a309b87
   [junit4]   2> 17944 INFO  (coreLoadExecutor-39-thread-1-processing-n:127.0.0.1:59039_zo%2Fji) [n:127.0.0.1:59039_zo%2Fji c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_20A35177A4CD4759-001/shard-1-001/cores/collection1/data
   [junit4]   2> 17944 INFO  (coreLoadExecutor-39-thread-1-processing-n:127.0.0.1:59039_zo%2Fji) [n:127.0.0.1:59039_zo%2Fji c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_20A35177A4CD4759-001/shard-1-001/cores/collection1/data/index/
   [junit4]   2> 17944 WARN  (coreLoadExecutor-39-thread-1-processing-n:127.0.0.1:59039_zo%2Fji) [n:127.0.0.1:59039_zo%2Fji c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_20A35177A4CD4759-001/shard-1-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 17945 INFO  (coreLoadExecutor-39-thread-1-processing-n:127.0.0.1:59039_zo%2Fji) [n:127.0.0.1:59039_zo%2Fji c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_20A35177A4CD4759-001/shard-1-001/cores/collection1/data/index
   [junit4]   2> 17945 INFO  (coreLoadExecutor-39-thread-1-processing-n:127.0.0.1:59039_zo%2Fji) [n:127.0.0.1:59039_zo%2Fji c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=614165572, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
   [junit4]   2> 17946 INFO  (coreLoadExecutor-39-thread-1-processing

[...truncated too long message...]

udSolrClient.request(CloudSolrClient.java:806)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1573)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1594)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:880)Throwable #3: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=665, name=collection2, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    > Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:44332/zo/ji: collection already exists: awholynewstresscollection_collection2_1
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([20A35177A4CD4759]:0)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:576)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:240)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:229)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1573)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1594)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:880)Throwable #4: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=663, name=collection0, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    > Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:44332/zo/ji: collection already exists: awholynewstresscollection_collection0_1
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([20A35177A4CD4759]:0)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:576)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:240)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:229)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1573)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1594)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:880)Throwable #5: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=667, name=collection4, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    > Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:44332/zo/ji: Could not find collection : awholynewstresscollection_collection4_1
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([20A35177A4CD4759]:0)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:576)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:240)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:229)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:887)Throwable #6: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=666, name=collection3, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    > Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:44332/zo/ji: Could not find collection : awholynewstresscollection_collection3_1
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([20A35177A4CD4759]:0)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:576)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:240)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:229)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:887)
   [junit4]   2> 418436 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[20A35177A4CD4759]-worker) [n:127.0.0.1:38952_zo%2Fji c:addReplicaColl s:shard1 r:core_node3 x:addReplicaColl_shard1_replica1] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> Feb 07, 2016 3:51:16 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=HighCompressionCompressingStoredFields(storedFieldsFormat=CompressingStoredFieldsFormat(compressionMode=HIGH_COMPRESSION, chunkSize=11153, maxDocsPerChunk=5, blockSize=440), termVectorsFormat=CompressingTermVectorsFormat(compressionMode=HIGH_COMPRESSION, chunkSize=11153, blockSize=440)), sim=DefaultSimilarity, locale=bg-BG, timezone=America/El_Salvador
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_80 (64-bit)/cpus=4,threads=1,free=88754192,total=488112128
   [junit4]   2> NOTE: All tests run in this JVM: [UpdateRequestProcessorFactoryTest, TestDynamicFieldCollectionResource, CollectionsAPIDistributedZkTest]
   [junit4] Completed [46/566 (1!)] on J1 in 409.99s, 1 test, 1 error <<< FAILURES!

[...truncated 1688 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:757: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:694: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:534: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1477: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1033: There were test failures: 566 suites (7 ignored), 2268 tests, 1 error, 33 ignored (18 assumptions) [seed: 20A35177A4CD4759]

Total time: 202 minutes 34 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any